[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:03:36 dist-test-slave-49h3 krb5kdc[26129](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:03:36 dist-test-slave-49h3 krb5kdc[26129](info): set up 2 sockets
Aug 12 19:03:36 dist-test-slave-49h3 krb5kdc[26129](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 19:03:37.103209 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:03:37 dist-test-slave-49h3 krb5kdc[26129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:03:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:03:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:03:37.234612 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:40471
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:45125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:40471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:37.351599 26147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:37.351897 26147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:37.351955 26147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:37.356101 26147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:37.356177 26147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:37.356206 26147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:37.356226 26147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:37.356248 26147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:37.361941 26147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:45125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:40471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:40471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:37.363166 26147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:37.364240 26147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:37.373391 26153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:37.373478 26147 server_base.cc:1034] running on GCE node
W20260812 19:03:37.373426 26156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:37.373377 26154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:37.374210 26147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:37.375357 26147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:37.376525 26147 hybrid_clock.cc:648] HybridClock initialized: now 1786561417376505 us; error 37 us; skew 500 ppm
Aug 12 19:03:37 dist-test-slave-49h3 krb5kdc[26129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561417, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:37.379451 26147 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:03:37.380448 26147 webserver.cc:460] Webserver started at http://127.25.130.62:41111/ using document root <none> and password file <none>
I20260812 19:03:37.381055 26147 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:37.381130 26147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:37.381346 26147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:37.383651 26147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "251d73f6af9e49eebe49801376e0d080"
format_stamp: "Formatted at 2026-08-12 19:03:37 on dist-test-slave-49h3"
server_key: "71ec26a07e382734aaa397a279e90d0d"
server_key_iv: "3b6d8208b3ec8d78649570763c3581ac"
server_key_version: "encryptionkey@0"
I20260812 19:03:37.384255 26147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "251d73f6af9e49eebe49801376e0d080"
format_stamp: "Formatted at 2026-08-12 19:03:37 on dist-test-slave-49h3"
server_key: "71ec26a07e382734aaa397a279e90d0d"
server_key_iv: "3b6d8208b3ec8d78649570763c3581ac"
server_key_version: "encryptionkey@0"
I20260812 19:03:37.388434 26147 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 19:03:37.391275 26164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:37.392765 26147 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 19:03:37.392970 26147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "251d73f6af9e49eebe49801376e0d080"
format_stamp: "Formatted at 2026-08-12 19:03:37 on dist-test-slave-49h3"
server_key: "71ec26a07e382734aaa397a279e90d0d"
server_key_iv: "3b6d8208b3ec8d78649570763c3581ac"
server_key_version: "encryptionkey@0"
I20260812 19:03:37.393116 26147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:37.427241 26147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:37.430711 26147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:37.430936 26147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:37.439448 26147 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:40471
I20260812 19:03:37.439762 26234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:40471 every 8 connection(s)
I20260812 19:03:37.440608 26147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:37.442957 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 26147
I20260812 19:03:37.443096 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:03:37.443852 26235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:37.444509 26120 external_mini_cluster.cc:1428] Setting key 5bc60c8a54120d1e8089bd8853c32727
I20260812 19:03:37.450222 26235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 251d73f6af9e49eebe49801376e0d080: Bootstrap starting.
I20260812 19:03:37.452826 26235 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 251d73f6af9e49eebe49801376e0d080: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:37.453852 26235 log.cc:824] T 00000000000000000000000000000000 P 251d73f6af9e49eebe49801376e0d080: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:37.456039 26235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 251d73f6af9e49eebe49801376e0d080: No bootstrap required, opened a new log
I20260812 19:03:37.460834 26235 raft_consensus.cc:348] T 00000000000000000000000000000000 P 251d73f6af9e49eebe49801376e0d080 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "251d73f6af9e49eebe49801376e0d080" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40471 } }
I20260812 19:03:37.461189 26235 raft_consensus.cc:374] T 00000000000000000000000000000000 P 251d73f6af9e49eebe49801376e0d080 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:37.461261 26235 raft_consensus.cc:729] T 00000000000000000000000000000000 P 251d73f6af9e49eebe49801376e0d080 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 251d73f6af9e49eebe49801376e0d080, State: Initialized, Role: FOLLOWER
I20260812 19:03:37.461793 26235 consensus_queue.cc:260] T 00000000000000000000000000000000 P 251d73f6af9e49eebe49801376e0d080 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "251d73f6af9e49eebe49801376e0d080" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40471 } }
I20260812 19:03:37.461939 26235 raft_consensus.cc:388] T 00000000000000000000000000000000 P 251d73f6af9e49eebe49801376e0d080 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:03:37.462004 26235 raft_consensus.cc:482] T 00000000000000000000000000000000 P 251d73f6af9e49eebe49801376e0d080 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:03:37.462132 26235 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 251d73f6af9e49eebe49801376e0d080 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:37.463271 26235 raft_consensus.cc:504] T 00000000000000000000000000000000 P 251d73f6af9e49eebe49801376e0d080 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "251d73f6af9e49eebe49801376e0d080" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40471 } }
Aug 12 19:03:37 dist-test-slave-49h3 krb5kdc[26129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:37.463696 26235 leader_election.cc:302] T 00000000000000000000000000000000 P 251d73f6af9e49eebe49801376e0d080 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 251d73f6af9e49eebe49801376e0d080; no voters:
I20260812 19:03:37.464078 26235 leader_election.cc:288] T 00000000000000000000000000000000 P 251d73f6af9e49eebe49801376e0d080 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:03:37.464228 26240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 251d73f6af9e49eebe49801376e0d080 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:37.464495 26240 raft_consensus.cc:686] T 00000000000000000000000000000000 P 251d73f6af9e49eebe49801376e0d080 [term 1 LEADER]: Becoming Leader. State: Replica: 251d73f6af9e49eebe49801376e0d080, State: Running, Role: LEADER
I20260812 19:03:37.464895 26240 consensus_queue.cc:237] T 00000000000000000000000000000000 P 251d73f6af9e49eebe49801376e0d080 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "251d73f6af9e49eebe49801376e0d080" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40471 } }
I20260812 19:03:37.465642 26235 sys_catalog.cc:564] T 00000000000000000000000000000000 P 251d73f6af9e49eebe49801376e0d080 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:37.466806 26241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 251d73f6af9e49eebe49801376e0d080 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "251d73f6af9e49eebe49801376e0d080" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "251d73f6af9e49eebe49801376e0d080" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40471 } } }
I20260812 19:03:37.466949 26241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 251d73f6af9e49eebe49801376e0d080 [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:37.467339 26242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 251d73f6af9e49eebe49801376e0d080 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 251d73f6af9e49eebe49801376e0d080. Latest consensus state: current_term: 1 leader_uuid: "251d73f6af9e49eebe49801376e0d080" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "251d73f6af9e49eebe49801376e0d080" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40471 } } }
I20260812 19:03:37.468534 26242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 251d73f6af9e49eebe49801376e0d080 [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:37.468987 26251 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:37.471421 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:37.447777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41896 (local address 127.25.130.62:40471)
0812 19:03:37.448323 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:37.448350 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:37.449413 (+ 1063us) server_negotiation.cc:407] Connection header received
0812 19:03:37.452830 (+ 3417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:37.452854 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:37.453309 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:37.453723 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:37.455974 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:37.457494 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:37.458911 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:37.459760 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:37.459952 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:37.464536 (+ 4584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:37.464545 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:37.464567 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:37.464613 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:37.467004 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:37.467811 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:37.467819 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:37.467825 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:37.467958 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:37.468390 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:37.468394 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:37.468396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:37.469143 (+ 747us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:37.469358 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:37.470167 (+ 809us) server_negotiation.cc:299] Negotiation successful
0812 19:03:37.470468 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":147,"threads_started":1}
I20260812 19:03:37.472689 26251 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:03:37.478837 26251 catalog_manager.cc:1347] Generated new cluster ID: 6f272fe10b3f4e72b123486a8984ed29
I20260812 19:03:37.478916 26251 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:37.490504 26251 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:37.491777 26251 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:37.500406 26251 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 251d73f6af9e49eebe49801376e0d080: Generated new TSK 0
I20260812 19:03:37.501148 26251 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
I20260812 19:03:37.540184 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:0
--local_ip_for_outbound_sockets=127.25.130.1
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:40471
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:45125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:37.662472 26269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:37.662783 26269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:37.662830 26269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:37.667052 26269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:37.667125 26269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:37.667196 26269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:03:37.672852 26269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:45125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:40471
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:37.674113 26269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:37.675218 26269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:37.685708 26281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:37.685659 26278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:37.685660 26275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:37.685700 26269 server_base.cc:1034] running on GCE node
I20260812 19:03:37.686426 26269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:37.687110 26269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:37.688282 26269 hybrid_clock.cc:648] HybridClock initialized: now 1786561417688248 us; error 48 us; skew 500 ppm
Aug 12 19:03:37 dist-test-slave-49h3 krb5kdc[26129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561417, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:37.691200 26269 init.cc:377] Logged in from keytab as kudu/127.25.130.1@KRBTEST.COM (short username kudu)
I20260812 19:03:37.692132 26269 webserver.cc:460] Webserver started at http://127.25.130.1:45199/ using document root <none> and password file <none>
I20260812 19:03:37.692682 26269 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:37.692752 26269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:37.692960 26269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:37.695087 26269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-0/data/instance:
uuid: "662f0c57d3d44cfd9022536f167c57bb"
format_stamp: "Formatted at 2026-08-12 19:03:37 on dist-test-slave-49h3"
server_key: "b3e9f1180bb728306e79e6673771cea3"
server_key_iv: "0a090f927678e5cfaea2df69dedaa703"
server_key_version: "encryptionkey@0"
I20260812 19:03:37.695647 26269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance:
uuid: "662f0c57d3d44cfd9022536f167c57bb"
format_stamp: "Formatted at 2026-08-12 19:03:37 on dist-test-slave-49h3"
server_key: "b3e9f1180bb728306e79e6673771cea3"
server_key_iv: "0a090f927678e5cfaea2df69dedaa703"
server_key_version: "encryptionkey@0"
I20260812 19:03:37.699695 26269 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:03:37.702246 26291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:37.703593 26269 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:03:37.703723 26269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "662f0c57d3d44cfd9022536f167c57bb"
format_stamp: "Formatted at 2026-08-12 19:03:37 on dist-test-slave-49h3"
server_key: "b3e9f1180bb728306e79e6673771cea3"
server_key_iv: "0a090f927678e5cfaea2df69dedaa703"
server_key_version: "encryptionkey@0"
I20260812 19:03:37.703855 26269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:37.716882 26269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:37.719901 26269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:37.720112 26269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:37.720760 26269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:03:37.723512 26269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:03:37.723809 26269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:37.724985 26269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:03:37.725797 26269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.003s
I20260812 19:03:37.735841 26269 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:45407
I20260812 19:03:37.735873 26437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:45407 every 8 connection(s)
I20260812 19:03:37.736842 26269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:03:37.738047 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 26269
I20260812 19:03:37.738145 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance
I20260812 19:03:37.738425 26120 external_mini_cluster.cc:1428] Setting key 99c3db32219d021a4453cc4d1d5be489
Aug 12 19:03:37 dist-test-slave-49h3 krb5kdc[26129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561417, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.130.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.2@KRBTEST.COM" created.
I20260812 19:03:37.751708 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:37.739211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:43529 (local address 127.25.130.62:40471)
0812 19:03:37.739362 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:37.739367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:37.740593 (+ 1226us) server_negotiation.cc:407] Connection header received
0812 19:03:37.741520 (+ 927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:37.741523 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:37.741583 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:37.741671 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:37.742907 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:37.743459 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:37.744676 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:37.744924 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:37.745017 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:37.747949 (+ 2932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:37.747954 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:37.747957 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:37.747993 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:37.749503 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:37.750044 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:37.750046 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:37.750048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:37.750129 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:37.750448 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:37.750450 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:37.750451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:37.750671 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:37.750764 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:37.751372 (+ 608us) server_negotiation.cc:299] Negotiation successful
0812 19:03:37.751528 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 19:03:37.752614 26441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:37.739648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:40471 (local address 127.25.130.1:43529)
0812 19:03:37.740409 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:37.740469 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:37.741291 (+ 822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:37.741840 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:37.741847 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:37.742228 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:37.742738 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:37.742753 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:37.743591 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:37.743595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:37.744534 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:37.744545 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:37.745086 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:37.745091 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:37.745291 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:37.745971 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:37.745988 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:37.747726 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:37.749642 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:37.749648 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:37.749662 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:37.749955 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:37.750214 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:37.750217 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:37.750218 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:37.750378 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:37.750779 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:37.750785 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:37.751098 (+ 313us) client_negotiation.cc:769] Sending connection context
0812 19:03:37.751284 (+ 186us) client_negotiation.cc:240] Negotiation successful
0812 19:03:37.751630 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":138,"threads_started":1}
I20260812 19:03:37.754058 26439 heartbeater.cc:347] Connected to a master server at 127.25.130.62:40471
I20260812 19:03:37.754349 26439 heartbeater.cc:464] Registering TS with master...
I20260812 19:03:37.754908 26439 heartbeater.cc:511] Master 127.25.130.62:40471 requested a full tablet report, sending...
I20260812 19:03:37.756690 26186 ts_manager.cc:194] Registered new tserver with Master: 662f0c57d3d44cfd9022536f167c57bb (127.25.130.1:45407)
I20260812 19:03:37.757951 26186 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.1@KRBTEST.COM'} at 127.25.130.1:43529
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
I20260812 19:03:37.766417 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:0
--local_ip_for_outbound_sockets=127.25.130.2
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:40471
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:45125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:37.882041 26445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:37.882326 26445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:37.882391 26445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:37.886729 26445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:37.886826 26445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:37.886920 26445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:03:37.892940 26445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:45125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:40471
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:37.894160 26445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:37.895224 26445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:37.905941 26460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:37.905928 26456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:37.905931 26453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:37.906430 26445 server_base.cc:1034] running on GCE node
I20260812 19:03:37.906975 26445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:37.907696 26445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:37.908861 26445 hybrid_clock.cc:648] HybridClock initialized: now 1786561417908823 us; error 56 us; skew 500 ppm
Aug 12 19:03:37 dist-test-slave-49h3 krb5kdc[26129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561417, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:37.912412 26445 init.cc:377] Logged in from keytab as kudu/127.25.130.2@KRBTEST.COM (short username kudu)
I20260812 19:03:37.913759 26445 webserver.cc:460] Webserver started at http://127.25.130.2:33151/ using document root <none> and password file <none>
I20260812 19:03:37.914351 26445 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:37.914422 26445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:37.914630 26445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:37.916795 26445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-1/data/instance:
uuid: "5b5ab990cd954ccda3ddf504df5fd148"
format_stamp: "Formatted at 2026-08-12 19:03:37 on dist-test-slave-49h3"
server_key: "93ce25972d2d4741457e860fd7018da2"
server_key_iv: "9864c617da0bef31c5640eb14b416a82"
server_key_version: "encryptionkey@0"
I20260812 19:03:37.917411 26445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance:
uuid: "5b5ab990cd954ccda3ddf504df5fd148"
format_stamp: "Formatted at 2026-08-12 19:03:37 on dist-test-slave-49h3"
server_key: "93ce25972d2d4741457e860fd7018da2"
server_key_iv: "9864c617da0bef31c5640eb14b416a82"
server_key_version: "encryptionkey@0"
I20260812 19:03:37.921698 26445 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 19:03:37.924443 26469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:37.925959 26445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:03:37.926122 26445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "5b5ab990cd954ccda3ddf504df5fd148"
format_stamp: "Formatted at 2026-08-12 19:03:37 on dist-test-slave-49h3"
server_key: "93ce25972d2d4741457e860fd7018da2"
server_key_iv: "9864c617da0bef31c5640eb14b416a82"
server_key_version: "encryptionkey@0"
I20260812 19:03:37.926247 26445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:37.937821 26445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:37.941723 26445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:37.941978 26445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:37.942672 26445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:03:37.943694 26445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:03:37.943745 26445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:37.943801 26445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:03:37.943832 26445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:37.952461 26445 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:42449
I20260812 19:03:37.952507 26610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:42449 every 8 connection(s)
I20260812 19:03:37.953395 26445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:03:37.954020 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 26445
I20260812 19:03:37.954138 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance
I20260812 19:03:37.954399 26120 external_mini_cluster.cc:1428] Setting key b9e40fbd07076d6b6f54ac25fd2ba788
Aug 12 19:03:37 dist-test-slave-49h3 krb5kdc[26129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561417, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.130.3@KRBTEST.COM; defaulting to no policy
I20260812 19:03:37.967432 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:37.955550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:35679 (local address 127.25.130.62:40471)
0812 19:03:37.955855 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:37.955860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:37.957022 (+ 1162us) server_negotiation.cc:407] Connection header received
0812 19:03:37.957960 (+ 938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:37.957963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:37.958023 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:37.958096 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:37.959413 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:37.959947 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:37.961081 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:37.961304 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:37.961394 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:37.963830 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:37.963833 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:37.963835 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:37.963862 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:37.965170 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:37.965754 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:37.965758 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:37.965759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:37.965846 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:37.966216 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:37.966218 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:37.966220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:37.966455 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:37.966545 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:37.967097 (+ 552us) server_negotiation.cc:299] Negotiation successful
0812 19:03:37.967236 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.3@KRBTEST.COM" created.
I20260812 19:03:37.968251 26614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:37.956141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:40471 (local address 127.25.130.2:35679)
0812 19:03:37.956867 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:37.956929 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:37.957760 (+ 831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:37.958290 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:37.958298 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:37.958676 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:37.959267 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:37.959283 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:37.960076 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:37.960078 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:37.960959 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:37.960969 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:37.961481 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:37.961485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:37.961663 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:37.962215 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:37.962233 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:37.963665 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:37.965285 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:37.965291 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:37.965308 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:37.965657 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:37.965937 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:37.965940 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:37.965942 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:37.966096 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:37.966557 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:37.966564 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:37.966832 (+ 268us) client_negotiation.cc:769] Sending connection context
0812 19:03:37.967013 (+ 181us) client_negotiation.cc:240] Negotiation successful
0812 19:03:37.967372 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":151,"threads_started":1}
I20260812 19:03:37.969501 26612 heartbeater.cc:347] Connected to a master server at 127.25.130.62:40471
I20260812 19:03:37.969787 26612 heartbeater.cc:464] Registering TS with master...
I20260812 19:03:37.970381 26612 heartbeater.cc:511] Master 127.25.130.62:40471 requested a full tablet report, sending...
I20260812 19:03:37.971596 26186 ts_manager.cc:194] Registered new tserver with Master: 5b5ab990cd954ccda3ddf504df5fd148 (127.25.130.2:42449)
I20260812 19:03:37.972227 26186 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.2@KRBTEST.COM'} at 127.25.130.2:35679
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
I20260812 19:03:37.984002 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:0
--local_ip_for_outbound_sockets=127.25.130.3
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:40471
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:45125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:38.098717 26619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:38.099007 26619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:38.099061 26619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:38.103215 26619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:38.103286 26619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:38.103349 26619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:03:38.108870 26619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:45125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:40471
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:38.110004 26619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:38.111078 26619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:38.120870 26628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:38.120954 26631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:38.120874 26627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:38.121312 26619 server_base.cc:1034] running on GCE node
I20260812 19:03:38.121762 26619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:38.122393 26619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:38.123565 26619 hybrid_clock.cc:648] HybridClock initialized: now 1786561418123545 us; error 33 us; skew 500 ppm
Aug 12 19:03:38 dist-test-slave-49h3 krb5kdc[26129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561418, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:38.126569 26619 init.cc:377] Logged in from keytab as kudu/127.25.130.3@KRBTEST.COM (short username kudu)
I20260812 19:03:38.127528 26619 webserver.cc:460] Webserver started at http://127.25.130.3:37795/ using document root <none> and password file <none>
I20260812 19:03:38.128109 26619 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:38.128178 26619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:38.128367 26619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:38.130988 26619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-2/data/instance:
uuid: "55ee10571f5e47c6b46dd322287174ad"
format_stamp: "Formatted at 2026-08-12 19:03:38 on dist-test-slave-49h3"
server_key: "5e773d2c760ac445016080c0a3e0c7a3"
server_key_iv: "9a28a928f5295cfceca3063766775da3"
server_key_version: "encryptionkey@0"
I20260812 19:03:38.131632 26619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance:
uuid: "55ee10571f5e47c6b46dd322287174ad"
format_stamp: "Formatted at 2026-08-12 19:03:38 on dist-test-slave-49h3"
server_key: "5e773d2c760ac445016080c0a3e0c7a3"
server_key_iv: "9a28a928f5295cfceca3063766775da3"
server_key_version: "encryptionkey@0"
I20260812 19:03:38.136075 26619 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:03:38.139083 26639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:38.140663 26619 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 19:03:38.140808 26619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "55ee10571f5e47c6b46dd322287174ad"
format_stamp: "Formatted at 2026-08-12 19:03:38 on dist-test-slave-49h3"
server_key: "5e773d2c760ac445016080c0a3e0c7a3"
server_key_iv: "9a28a928f5295cfceca3063766775da3"
server_key_version: "encryptionkey@0"
I20260812 19:03:38.140941 26619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:38.160029 26619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:38.164412 26619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:38.164645 26619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:38.165310 26619 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:03:38.166435 26619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:03:38.166495 26619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:38.166577 26619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:03:38.166607 26619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:38.177186 26619 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:39295
I20260812 19:03:38.177246 26787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:39295 every 8 connection(s)
I20260812 19:03:38.178229 26619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:03:38.181819 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 26619
I20260812 19:03:38.181954 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance
I20260812 19:03:38.182258 26120 external_mini_cluster.cc:1428] Setting key 745d17065c20ee6f2b4aaaea89caed89
Aug 12 19:03:38 dist-test-slave-49h3 krb5kdc[26129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561418, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:38.192360 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:38.180360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:55561 (local address 127.25.130.62:40471)
0812 19:03:38.180545 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:38.180551 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:38.181777 (+ 1226us) server_negotiation.cc:407] Connection header received
0812 19:03:38.182962 (+ 1185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:38.182965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:38.183020 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:38.183088 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:38.184554 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:38.185218 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:38.186412 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:38.186625 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:38.186710 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:38.189032 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:38.189035 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:38.189037 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:38.189063 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:38.190434 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:38.190909 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:38.190912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:38.190913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:38.190994 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:38.191310 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:38.191313 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:38.191314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:38.191495 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:38.191580 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:38.192071 (+ 491us) server_negotiation.cc:299] Negotiation successful
0812 19:03:38.192194 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 19:03:38.193181 26792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:38.180772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:40471 (local address 127.25.130.3:55561)
0812 19:03:38.181592 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:38.181654 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:38.182721 (+ 1067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:38.183282 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:38.183292 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:38.183828 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:38.184400 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:38.184413 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:38.185337 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:38.185340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:38.186304 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:38.186314 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:38.186748 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:38.186750 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:38.186895 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:38.187438 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:38.187455 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:38.188877 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:38.190543 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:38.190549 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:38.190564 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:38.190817 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:38.191074 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:38.191077 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:38.191079 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:38.191224 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:38.191593 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:38.191600 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:38.191843 (+ 243us) client_negotiation.cc:769] Sending connection context
0812 19:03:38.192010 (+ 167us) client_negotiation.cc:240] Negotiation successful
0812 19:03:38.192350 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":134,"threads_started":1}
I20260812 19:03:38.194276 26790 heartbeater.cc:347] Connected to a master server at 127.25.130.62:40471
I20260812 19:03:38.194519 26790 heartbeater.cc:464] Registering TS with master...
I20260812 19:03:38.195027 26790 heartbeater.cc:511] Master 127.25.130.62:40471 requested a full tablet report, sending...
I20260812 19:03:38.196062 26185 ts_manager.cc:194] Registered new tserver with Master: 55ee10571f5e47c6b46dd322287174ad (127.25.130.3:39295)
I20260812 19:03:38.196755 26185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.3@KRBTEST.COM'} at 127.25.130.3:55561
I20260812 19:03:38.196878 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 19:03:38 dist-test-slave-49h3 krb5kdc[26129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561418, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 19:03:38 dist-test-slave-49h3 krb5kdc[26129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561418, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.130.1@KRBTEST.COM
I20260812 19:03:38.225697 26800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:38.215791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49966 (local address 127.25.130.1:45407)
0812 19:03:38.216096 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:38.216113 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:38.216137 (+ 24us) server_negotiation.cc:407] Connection header received
0812 19:03:38.216191 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:38.216196 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:38.216394 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:38.216512 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:38.217270 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:38.218143 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:38.219025 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:38.219416 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:38.219493 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:38.222013 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:38.222027 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:38.222043 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:38.222074 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:38.223714 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:38.224229 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:38.224235 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:38.224237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:38.224339 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:38.224560 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:38.224562 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:38.224563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:38.224838 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:38.225127 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:38.225307 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 19:03:38.225497 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":91,"threads_started":1}
W20260812 19:03:38.226698 26325 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49966
Aug 12 19:03:38 dist-test-slave-49h3 krb5kdc[26129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 19:03:38 dist-test-slave-49h3 krb5kdc[26129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.1@KRBTEST.COM
I20260812 19:03:38.254621 26800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:38.246454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49970 (local address 127.25.130.1:45407)
0812 19:03:38.246634 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:38.246637 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:38.246720 (+ 83us) server_negotiation.cc:407] Connection header received
0812 19:03:38.246838 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:38.246842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:38.246887 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:38.246955 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:38.247641 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:38.248167 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:38.249007 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:38.249227 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:38.249294 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:38.251673 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:38.251676 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:38.251678 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:38.251717 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:38.253025 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:38.253496 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:38.253499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:38.253501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:38.253590 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:38.253818 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:38.253819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:38.253821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:38.253993 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:38.254091 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:38.254349 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 19:03:38.254457 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 19:03:38.255721 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 26269
I20260812 19:03:38.264153 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 26445
I20260812 19:03:38.272569 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 26619
I20260812 19:03:38.281133 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 26147
2026-08-12T19:03:38Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2288 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:03:38 dist-test-slave-49h3 krb5kdc[26809](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:03:38 dist-test-slave-49h3 krb5kdc[26809](info): set up 2 sockets
Aug 12 19:03:38 dist-test-slave-49h3 krb5kdc[26809](info): commencing operation
krb5kdc: starting...
W20260812 19:03:39.339231 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:03:39 dist-test-slave-49h3 krb5kdc[26809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:03:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:03:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:03:39.464341 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:42131
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:40883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:42131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:39.580811 26823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:39.581125 26823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:39.581187 26823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:39.585392 26823 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:39.585505 26823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:39.585541 26823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:39.585563 26823 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:39.585585 26823 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:39.591333 26823 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:40883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:42131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:42131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:39.592561 26823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:39.593698 26823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:39.602676 26829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:39.602705 26823 server_base.cc:1034] running on GCE node
W20260812 19:03:39.602661 26833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:39.602661 26830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:39.603533 26823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:39.604735 26823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:39.605916 26823 hybrid_clock.cc:648] HybridClock initialized: now 1786561419605900 us; error 29 us; skew 500 ppm
Aug 12 19:03:39 dist-test-slave-49h3 krb5kdc[26809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561419, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:39.608702 26823 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:03:39.609706 26823 webserver.cc:460] Webserver started at http://127.25.130.62:32999/ using document root <none> and password file <none>
I20260812 19:03:39.610286 26823 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:39.610361 26823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:39.610598 26823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:39.612879 26823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "0bc3993f2464443fb0cc66b52ca5b984"
format_stamp: "Formatted at 2026-08-12 19:03:39 on dist-test-slave-49h3"
server_key: "aea157e3cefde46bf7cdaef76c593359"
server_key_iv: "19e29a9ed0d18200dd8ebbd2dd7d8d1b"
server_key_version: "encryptionkey@0"
I20260812 19:03:39.613564 26823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "0bc3993f2464443fb0cc66b52ca5b984"
format_stamp: "Formatted at 2026-08-12 19:03:39 on dist-test-slave-49h3"
server_key: "aea157e3cefde46bf7cdaef76c593359"
server_key_iv: "19e29a9ed0d18200dd8ebbd2dd7d8d1b"
server_key_version: "encryptionkey@0"
I20260812 19:03:39.617767 26823 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:03:39.620568 26840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:39.622115 26823 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:03:39.622262 26823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "0bc3993f2464443fb0cc66b52ca5b984"
format_stamp: "Formatted at 2026-08-12 19:03:39 on dist-test-slave-49h3"
server_key: "aea157e3cefde46bf7cdaef76c593359"
server_key_iv: "19e29a9ed0d18200dd8ebbd2dd7d8d1b"
server_key_version: "encryptionkey@0"
I20260812 19:03:39.622387 26823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:39.631912 26823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:39.634955 26823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:39.635160 26823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:39.642441 26823 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:42131
I20260812 19:03:39.642483 26909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:42131 every 8 connection(s)
I20260812 19:03:39.643478 26823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:39.646560 26910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:39.651646 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 26823
I20260812 19:03:39.651747 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:03:39.651993 26120 external_mini_cluster.cc:1428] Setting key 848b7dc9e4d7ce41dde784dd46731973
I20260812 19:03:39.653751 26910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bc3993f2464443fb0cc66b52ca5b984: Bootstrap starting.
I20260812 19:03:39.656351 26910 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bc3993f2464443fb0cc66b52ca5b984: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:39.657121 26910 log.cc:824] T 00000000000000000000000000000000 P 0bc3993f2464443fb0cc66b52ca5b984: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:39.659395 26910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bc3993f2464443fb0cc66b52ca5b984: No bootstrap required, opened a new log
Aug 12 19:03:39 dist-test-slave-49h3 krb5kdc[26809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:39.664055 26910 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bc3993f2464443fb0cc66b52ca5b984 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bc3993f2464443fb0cc66b52ca5b984" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42131 } }
I20260812 19:03:39.664376 26910 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bc3993f2464443fb0cc66b52ca5b984 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:39.664444 26910 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bc3993f2464443fb0cc66b52ca5b984 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bc3993f2464443fb0cc66b52ca5b984, State: Initialized, Role: FOLLOWER
I20260812 19:03:39.664989 26910 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bc3993f2464443fb0cc66b52ca5b984 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bc3993f2464443fb0cc66b52ca5b984" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42131 } }
I20260812 19:03:39.665169 26910 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bc3993f2464443fb0cc66b52ca5b984 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:03:39.665247 26910 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bc3993f2464443fb0cc66b52ca5b984 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:03:39.665349 26910 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bc3993f2464443fb0cc66b52ca5b984 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:39.666688 26910 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bc3993f2464443fb0cc66b52ca5b984 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bc3993f2464443fb0cc66b52ca5b984" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42131 } }
I20260812 19:03:39.667074 26910 leader_election.cc:302] T 00000000000000000000000000000000 P 0bc3993f2464443fb0cc66b52ca5b984 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bc3993f2464443fb0cc66b52ca5b984; no voters:
I20260812 19:03:39.667450 26910 leader_election.cc:288] T 00000000000000000000000000000000 P 0bc3993f2464443fb0cc66b52ca5b984 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:03:39.667598 26914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:39.653622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60776 (local address 127.25.130.62:42131)
0812 19:03:39.654061 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:39.654077 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:39.654115 (+ 38us) server_negotiation.cc:407] Connection header received
0812 19:03:39.654803 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:39.654821 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:39.655170 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:39.655456 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:39.656391 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:39.657559 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:39.658503 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:39.659201 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:39.659368 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:39.662034 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:39.662041 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:39.662061 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:39.662102 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:39.664117 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:39.664619 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:39.664625 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:39.664630 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:39.664786 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:39.665052 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:39.665055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:39.665057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:39.665704 (+ 647us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:39.665909 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:39.666173 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 19:03:39.666513 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":124,"threads_started":1}
I20260812 19:03:39.667573 26917 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bc3993f2464443fb0cc66b52ca5b984 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:39.667871 26917 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bc3993f2464443fb0cc66b52ca5b984 [term 1 LEADER]: Becoming Leader. State: Replica: 0bc3993f2464443fb0cc66b52ca5b984, State: Running, Role: LEADER
I20260812 19:03:39.668202 26917 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bc3993f2464443fb0cc66b52ca5b984 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bc3993f2464443fb0cc66b52ca5b984" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42131 } }
I20260812 19:03:39.668828 26910 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bc3993f2464443fb0cc66b52ca5b984 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:39.669407 26917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bc3993f2464443fb0cc66b52ca5b984 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bc3993f2464443fb0cc66b52ca5b984. Latest consensus state: current_term: 1 leader_uuid: "0bc3993f2464443fb0cc66b52ca5b984" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bc3993f2464443fb0cc66b52ca5b984" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42131 } } }
I20260812 19:03:39.669555 26917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bc3993f2464443fb0cc66b52ca5b984 [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:39.671000 26919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bc3993f2464443fb0cc66b52ca5b984 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bc3993f2464443fb0cc66b52ca5b984" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bc3993f2464443fb0cc66b52ca5b984" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42131 } } }
I20260812 19:03:39.671103 26919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bc3993f2464443fb0cc66b52ca5b984 [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:39.671439 26924 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:39.674603 26924 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:03:39.680831 26924 catalog_manager.cc:1347] Generated new cluster ID: a9656ef66d4a4b9cad3dfeccfdf6c226
I20260812 19:03:39.680903 26924 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:39.690842 26924 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:39.691846 26924 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:39.699601 26924 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bc3993f2464443fb0cc66b52ca5b984: Generated new TSK 0
I20260812 19:03:39.700400 26924 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
I20260812 19:03:39.730976 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:0
--local_ip_for_outbound_sockets=127.25.130.1
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:42131
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:40883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:39.849185 26943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:39.849542 26943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:39.849611 26943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:39.853864 26943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:39.853951 26943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:39.854040 26943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:03:39.859771 26943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:40883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:42131
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:39.860993 26943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:39.862150 26943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:39.872113 26948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:39.872114 26953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:39.872169 26949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:39.872437 26943 server_base.cc:1034] running on GCE node
I20260812 19:03:39.872902 26943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:39.873612 26943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:39.874780 26943 hybrid_clock.cc:648] HybridClock initialized: now 1786561419874759 us; error 35 us; skew 500 ppm
Aug 12 19:03:39 dist-test-slave-49h3 krb5kdc[26809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561419, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:39.877602 26943 init.cc:377] Logged in from keytab as kudu/127.25.130.1@KRBTEST.COM (short username kudu)
I20260812 19:03:39.878574 26943 webserver.cc:460] Webserver started at http://127.25.130.1:43903/ using document root <none> and password file <none>
I20260812 19:03:39.879160 26943 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:39.879230 26943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:39.879437 26943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:39.881673 26943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-0/data/instance:
uuid: "29504c919f314b049deb56fffcff0261"
format_stamp: "Formatted at 2026-08-12 19:03:39 on dist-test-slave-49h3"
server_key: "69686ad92704219e717dca09f88d5093"
server_key_iv: "0bcbcd45ee394fb2fb9eb5d7948c80aa"
server_key_version: "encryptionkey@0"
I20260812 19:03:39.882287 26943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance:
uuid: "29504c919f314b049deb56fffcff0261"
format_stamp: "Formatted at 2026-08-12 19:03:39 on dist-test-slave-49h3"
server_key: "69686ad92704219e717dca09f88d5093"
server_key_iv: "0bcbcd45ee394fb2fb9eb5d7948c80aa"
server_key_version: "encryptionkey@0"
I20260812 19:03:39.886365 26943 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:03:39.889005 26963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:39.890380 26943 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 19:03:39.890511 26943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "29504c919f314b049deb56fffcff0261"
format_stamp: "Formatted at 2026-08-12 19:03:39 on dist-test-slave-49h3"
server_key: "69686ad92704219e717dca09f88d5093"
server_key_iv: "0bcbcd45ee394fb2fb9eb5d7948c80aa"
server_key_version: "encryptionkey@0"
I20260812 19:03:39.890623 26943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:39.906885 26943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:39.910000 26943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:39.910199 26943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:39.910823 26943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:03:39.911713 26943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:03:39.911772 26943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:39.911829 26943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:03:39.911856 26943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:39.920979 26943 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:43281
I20260812 19:03:39.921067 27114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:43281 every 8 connection(s)
I20260812 19:03:39.922016 26943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:03:39.929008 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 26943
I20260812 19:03:39.929123 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance
I20260812 19:03:39.929389 26120 external_mini_cluster.cc:1428] Setting key 434240f30d2e0bb45b57e023d2a77ab9
Aug 12 19:03:39 dist-test-slave-49h3 krb5kdc[26809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561419, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:39.937510 26914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:39.924079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:46299 (local address 127.25.130.62:42131)
0812 19:03:39.924250 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:39.924254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:39.925153 (+ 899us) server_negotiation.cc:407] Connection header received
0812 19:03:39.926629 (+ 1476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:39.926632 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:39.926696 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:39.926768 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:39.928082 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:39.928629 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:39.929984 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:39.930282 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:39.930372 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:39.933061 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:39.933065 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:39.933067 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:39.933096 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:39.934458 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:39.935100 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:39.935104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:39.935106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:39.935240 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:39.935734 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:39.935738 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:39.935740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:39.936031 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:39.936140 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:39.936950 (+ 810us) server_negotiation.cc:299] Negotiation successful
0812 19:03:39.937204 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 19:03:39.938496 27119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:39.924441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:42131 (local address 127.25.130.1:46299)
0812 19:03:39.924983 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:39.925032 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:39.926377 (+ 1345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:39.926981 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:39.926989 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:39.927384 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:39.927912 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:39.927926 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:39.928768 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:39.928771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:39.929819 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:39.929834 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:39.930456 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:39.930460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:39.930677 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:39.931340 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:39.931359 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:39.932841 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:39.934589 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:39.934597 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:39.934617 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:39.934970 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:39.935365 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:39.935369 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:39.935371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:39.935616 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:39.936173 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:39.936182 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:39.936600 (+ 418us) client_negotiation.cc:769] Sending connection context
0812 19:03:39.936846 (+ 246us) client_negotiation.cc:240] Negotiation successful
0812 19:03:39.937267 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":132,"threads_started":1}
WARNING: no policy specified for kudu/127.25.130.2@KRBTEST.COM; defaulting to no policy
I20260812 19:03:39.940929 27117 heartbeater.cc:347] Connected to a master server at 127.25.130.62:42131
I20260812 19:03:39.941196 27117 heartbeater.cc:464] Registering TS with master...
I20260812 19:03:39.941804 27117 heartbeater.cc:511] Master 127.25.130.62:42131 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.2@KRBTEST.COM" created.
I20260812 19:03:39.943503 26861 ts_manager.cc:194] Registered new tserver with Master: 29504c919f314b049deb56fffcff0261 (127.25.130.1:43281)
I20260812 19:03:39.944708 26861 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.1@KRBTEST.COM'} at 127.25.130.1:46299
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
I20260812 19:03:39.960014 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:0
--local_ip_for_outbound_sockets=127.25.130.2
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:42131
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:40883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:40.077073 27122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:40.077382 27122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:40.077430 27122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:40.081689 27122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:40.081775 27122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:40.081843 27122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:03:40.088065 27122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:40883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:42131
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:40.089257 27122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:40.090356 27122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:40.100401 27128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:40.100401 27129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:40.100446 27122 server_base.cc:1034] running on GCE node
W20260812 19:03:40.100401 27131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:40.101145 27122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:40.101893 27122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:40.103084 27122 hybrid_clock.cc:648] HybridClock initialized: now 1786561420103052 us; error 45 us; skew 500 ppm
Aug 12 19:03:40 dist-test-slave-49h3 krb5kdc[26809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561420, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:40.106154 27122 init.cc:377] Logged in from keytab as kudu/127.25.130.2@KRBTEST.COM (short username kudu)
I20260812 19:03:40.107203 27122 webserver.cc:460] Webserver started at http://127.25.130.2:38463/ using document root <none> and password file <none>
I20260812 19:03:40.107800 27122 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:40.107877 27122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:40.108114 27122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:40.110404 27122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-1/data/instance:
uuid: "5bd3fc71996542bbbee59a45cd3f3200"
format_stamp: "Formatted at 2026-08-12 19:03:40 on dist-test-slave-49h3"
server_key: "3b69e4cc547c074e0df067fc77e83116"
server_key_iv: "b420175c48de6b57a7ab330354e93e4b"
server_key_version: "encryptionkey@0"
I20260812 19:03:40.111028 27122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance:
uuid: "5bd3fc71996542bbbee59a45cd3f3200"
format_stamp: "Formatted at 2026-08-12 19:03:40 on dist-test-slave-49h3"
server_key: "3b69e4cc547c074e0df067fc77e83116"
server_key_iv: "b420175c48de6b57a7ab330354e93e4b"
server_key_version: "encryptionkey@0"
I20260812 19:03:40.115247 27122 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:03:40.118013 27142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:40.119402 27122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:03:40.119539 27122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "5bd3fc71996542bbbee59a45cd3f3200"
format_stamp: "Formatted at 2026-08-12 19:03:40 on dist-test-slave-49h3"
server_key: "3b69e4cc547c074e0df067fc77e83116"
server_key_iv: "b420175c48de6b57a7ab330354e93e4b"
server_key_version: "encryptionkey@0"
I20260812 19:03:40.119655 27122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:40.132788 27122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:40.136255 27122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:40.136451 27122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:40.137084 27122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:03:40.138155 27122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:03:40.138216 27122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:40.138276 27122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:03:40.138306 27122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:40.146857 27122 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:37611
I20260812 19:03:40.146898 27282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:37611 every 8 connection(s)
I20260812 19:03:40.147799 27122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
Aug 12 19:03:40 dist-test-slave-49h3 krb5kdc[26809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561420, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:40.157812 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 27122
I20260812 19:03:40.157943 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance
I20260812 19:03:40.158228 26120 external_mini_cluster.cc:1428] Setting key 1143cee67e562d6427da4dd65dc21b3c
I20260812 19:03:40.162494 26914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:40.149761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:55647 (local address 127.25.130.62:42131)
0812 19:03:40.149907 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:40.149910 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:40.150811 (+ 901us) server_negotiation.cc:407] Connection header received
0812 19:03:40.151734 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:40.151738 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:40.151799 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:40.151876 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:40.153092 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.153706 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.155004 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.155246 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.155340 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:40.158096 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:40.158100 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:40.158103 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:40.158136 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:40.159808 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:40.160519 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:40.160522 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:40.160523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:40.160631 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:40.161042 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:40.161045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:40.161046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:40.161255 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:40.161334 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:40.162054 (+ 720us) server_negotiation.cc:299] Negotiation successful
0812 19:03:40.162318 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 19:03:40.163534 27285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:40.150168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:42131 (local address 127.25.130.2:55647)
0812 19:03:40.150656 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:40.150699 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:40.151515 (+ 816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:40.152065 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:40.152072 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:40.152447 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:40.152937 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:40.152951 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.153846 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.153849 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:40.154850 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:40.154862 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.155398 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.155402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:40.155565 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:40.156144 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:40.156161 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:40.157856 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:40.159950 (+ 2094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:40.159958 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:40.159977 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:40.160393 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:40.160736 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:40.160739 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:40.160741 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:40.160951 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:40.161355 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:40.161362 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:40.161750 (+ 388us) client_negotiation.cc:769] Sending connection context
0812 19:03:40.161975 (+ 225us) client_negotiation.cc:240] Negotiation successful
0812 19:03:40.162364 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":132,"threads_started":1}
I20260812 19:03:40.164786 27283 heartbeater.cc:347] Connected to a master server at 127.25.130.62:42131
I20260812 19:03:40.165037 27283 heartbeater.cc:464] Registering TS with master...
I20260812 19:03:40.165657 27283 heartbeater.cc:511] Master 127.25.130.62:42131 requested a full tablet report, sending...
I20260812 19:03:40.166795 26861 ts_manager.cc:194] Registered new tserver with Master: 5bd3fc71996542bbbee59a45cd3f3200 (127.25.130.2:37611)
I20260812 19:03:40.167389 26861 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.2@KRBTEST.COM'} at 127.25.130.2:55647
WARNING: no policy specified for kudu/127.25.130.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
I20260812 19:03:40.187786 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:0
--local_ip_for_outbound_sockets=127.25.130.3
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:42131
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:40883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:40.305186 27291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:40.305518 27291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:40.305613 27291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:40.309772 27291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:40.309845 27291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:40.309928 27291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:03:40.315507 27291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:40883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:42131
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:40.316712 27291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:40.317866 27291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:40.327991 27299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:40.328011 27296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:40.328011 27297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:40.328197 27291 server_base.cc:1034] running on GCE node
I20260812 19:03:40.328696 27291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:40.329416 27291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:40.330628 27291 hybrid_clock.cc:648] HybridClock initialized: now 1786561420330604 us; error 33 us; skew 500 ppm
Aug 12 19:03:40 dist-test-slave-49h3 krb5kdc[26809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561420, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:40.333597 27291 init.cc:377] Logged in from keytab as kudu/127.25.130.3@KRBTEST.COM (short username kudu)
I20260812 19:03:40.334568 27291 webserver.cc:460] Webserver started at http://127.25.130.3:42577/ using document root <none> and password file <none>
I20260812 19:03:40.335157 27291 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:40.335222 27291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:40.335403 27291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:40.337617 27291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-2/data/instance:
uuid: "b63ba57646f740cc91c66d269496ec19"
format_stamp: "Formatted at 2026-08-12 19:03:40 on dist-test-slave-49h3"
server_key: "e7b83d4e931dcf8e9bee0431abca04c8"
server_key_iv: "463e3965aea698c05fdc6f428a3b3255"
server_key_version: "encryptionkey@0"
I20260812 19:03:40.338199 27291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance:
uuid: "b63ba57646f740cc91c66d269496ec19"
format_stamp: "Formatted at 2026-08-12 19:03:40 on dist-test-slave-49h3"
server_key: "e7b83d4e931dcf8e9bee0431abca04c8"
server_key_iv: "463e3965aea698c05fdc6f428a3b3255"
server_key_version: "encryptionkey@0"
I20260812 19:03:40.342202 27291 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:03:40.344827 27306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:40.346225 27291 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:03:40.346345 27291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "b63ba57646f740cc91c66d269496ec19"
format_stamp: "Formatted at 2026-08-12 19:03:40 on dist-test-slave-49h3"
server_key: "e7b83d4e931dcf8e9bee0431abca04c8"
server_key_iv: "463e3965aea698c05fdc6f428a3b3255"
server_key_version: "encryptionkey@0"
I20260812 19:03:40.346449 27291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:40.363019 27291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:40.366391 27291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:40.366564 27291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:40.367152 27291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:03:40.368031 27291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:03:40.368083 27291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:40.368136 27291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:03:40.368158 27291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:40.377281 27291 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:45925
I20260812 19:03:40.377319 27448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:45925 every 8 connection(s)
I20260812 19:03:40.378306 27291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:03:40.385703 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 27291
I20260812 19:03:40.385844 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance
I20260812 19:03:40.386158 26120 external_mini_cluster.cc:1428] Setting key cd921764b937e5a4b1c42e1b81e02ee2
Aug 12 19:03:40 dist-test-slave-49h3 krb5kdc[26809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561420, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:40.393936 26914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:40.380332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:43251 (local address 127.25.130.62:42131)
0812 19:03:40.380480 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:40.380484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:40.381273 (+ 789us) server_negotiation.cc:407] Connection header received
0812 19:03:40.382640 (+ 1367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:40.382643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:40.382708 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:40.382787 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:40.384108 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.384654 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.386022 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.386259 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.386366 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:40.389677 (+ 3311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:40.389680 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:40.389682 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:40.389728 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:40.391426 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:40.392049 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:40.392052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:40.392053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:40.392150 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:40.392561 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:40.392563 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:40.392565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:40.392789 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:40.392871 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:40.393653 (+ 782us) server_negotiation.cc:299] Negotiation successful
0812 19:03:40.393802 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 19:03:40.394896 27451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:40.380605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:42131 (local address 127.25.130.3:43251)
0812 19:03:40.381112 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:40.381160 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:40.382407 (+ 1247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:40.382996 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:40.383004 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:40.383394 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:40.383944 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:40.383960 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.384789 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.384792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:40.385877 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:40.385891 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.386427 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.386431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:40.386626 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:40.387486 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:40.387509 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:40.389440 (+ 1931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:40.391585 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:40.391591 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:40.391608 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:40.391927 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:40.392260 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:40.392263 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:40.392265 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:40.392443 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:40.392907 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:40.392914 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:40.393304 (+ 390us) client_negotiation.cc:769] Sending connection context
0812 19:03:40.393566 (+ 262us) client_negotiation.cc:240] Negotiation successful
0812 19:03:40.393934 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":121,"threads_started":1}
I20260812 19:03:40.396318 27449 heartbeater.cc:347] Connected to a master server at 127.25.130.62:42131
I20260812 19:03:40.396590 27449 heartbeater.cc:464] Registering TS with master...
I20260812 19:03:40.397146 27449 heartbeater.cc:511] Master 127.25.130.62:42131 requested a full tablet report, sending...
I20260812 19:03:40.398317 26861 ts_manager.cc:194] Registered new tserver with Master: b63ba57646f740cc91c66d269496ec19 (127.25.130.3:45925)
I20260812 19:03:40.398921 26861 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.3@KRBTEST.COM'} at 127.25.130.3:43251
I20260812 19:03:40.401063 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:03:40.412205 26914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:40.404947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60782 (local address 127.25.130.62:42131)
0812 19:03:40.405066 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:40.405069 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:40.405245 (+ 176us) server_negotiation.cc:407] Connection header received
0812 19:03:40.405379 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:40.405381 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:40.405426 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:40.405523 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:40.406225 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.406744 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.407567 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.407791 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.407872 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:40.409306 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:40.409309 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:40.409313 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:40.409343 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:40.410745 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:40.411175 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:40.411178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:40.411179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:40.411256 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:40.411449 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:40.411450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:40.411451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:40.411695 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:40.411799 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:40.411966 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 19:03:40.412069 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 19:03:40.417335 26861 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60782:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:03:40.420502 26861 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:03:40.437935 27464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:40.432245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:43281 (local address 127.0.0.1:45558)
0812 19:03:40.433235 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:40.433247 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:40.433328 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:40.433881 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:40.433884 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:40.433898 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:03:40.433982 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:40.433987 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.434949 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.434953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:40.436189 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:40.436200 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.437269 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.437273 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:40.437397 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:40.437468 (+ 71us) client_negotiation.cc:769] Sending connection context
0812 19:03:40.437607 (+ 139us) client_negotiation.cc:240] Negotiation successful
0812 19:03:40.437759 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":928,"thread_start_us":106,"threads_started":1}
I20260812 19:03:40.437945 27468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:40.432455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45558 (local address 127.25.130.1:43281)
0812 19:03:40.432783 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:40.432799 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:40.433421 (+ 622us) server_negotiation.cc:407] Connection header received
0812 19:03:40.433544 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:40.433547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:40.433723 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:40.433874 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:03:40.434084 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.434820 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.436320 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.437125 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.437245 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:40.437419 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:40.437658 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 19:03:40.437779 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":129,"threads_started":1}
I20260812 19:03:40.438506 27460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:40.431124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:37611 (local address 127.0.0.1:46338)
0812 19:03:40.431874 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:40.431924 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:40.432048 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:40.433331 (+ 1283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:40.433337 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:40.433365 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:03:40.433573 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:40.433583 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.435272 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.435276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:40.436812 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:40.436825 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.438109 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.438113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:40.438213 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:40.438239 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 19:03:40.438302 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 19:03:40.438378 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":108,"thread_start_us":196,"threads_started":1}
I20260812 19:03:40.438952 27467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:40.431257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:45925 (local address 127.0.0.1:50058)
0812 19:03:40.432854 (+ 1597us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:40.432868 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:40.433025 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:40.433631 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:40.433634 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:40.433653 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:03:40.433751 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:40.433757 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.435272 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.435276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:40.436817 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:40.436825 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.438346 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.438350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:40.438451 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:40.438476 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 19:03:40.438553 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 19:03:40.438819 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1535,"mutex_wait_us":111,"thread_start_us":119,"threads_started":1}
I20260812 19:03:40.439041 27465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:40.431267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46338 (local address 127.25.130.2:37611)
0812 19:03:40.432729 (+ 1462us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:40.432741 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:40.432768 (+ 27us) server_negotiation.cc:407] Connection header received
0812 19:03:40.432851 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:40.432856 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:40.433093 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:40.433261 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:03:40.433725 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.435101 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.436972 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.437794 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.437903 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:40.438040 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:40.438795 (+ 755us) server_negotiation.cc:299] Negotiation successful
0812 19:03:40.438895 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1358,"thread_start_us":167,"threads_started":1}
I20260812 19:03:40.439049 27466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:40.431704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50058 (local address 127.25.130.3:45925)
0812 19:03:40.432365 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:40.432383 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:40.432952 (+ 569us) server_negotiation.cc:407] Connection header received
0812 19:03:40.433160 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:40.433167 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:40.433476 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:40.433641 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:03:40.433850 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.435079 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.436955 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.437806 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.437917 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:40.438261 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:40.438788 (+ 527us) server_negotiation.cc:299] Negotiation successful
0812 19:03:40.438894 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":312,"threads_started":1}
I20260812 19:03:40.441586 27024 tablet_service.cc:1467] Processing CreateTablet for tablet 15a380b9e3284d8381a0cef8ed33e826 (DEFAULT_TABLE table=test-table [id=9d4e8a1579804ce7b87d5740d2da2771]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:03:40.441587 27202 tablet_service.cc:1467] Processing CreateTablet for tablet 15a380b9e3284d8381a0cef8ed33e826 (DEFAULT_TABLE table=test-table [id=9d4e8a1579804ce7b87d5740d2da2771]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:03:40.442140 27373 tablet_service.cc:1467] Processing CreateTablet for tablet 15a380b9e3284d8381a0cef8ed33e826 (DEFAULT_TABLE table=test-table [id=9d4e8a1579804ce7b87d5740d2da2771]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:03:40.442828 27024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15a380b9e3284d8381a0cef8ed33e826. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:40.442828 27202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15a380b9e3284d8381a0cef8ed33e826. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:40.443135 27373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15a380b9e3284d8381a0cef8ed33e826. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:40.450418 27469 tablet_bootstrap.cc:492] T 15a380b9e3284d8381a0cef8ed33e826 P b63ba57646f740cc91c66d269496ec19: Bootstrap starting.
I20260812 19:03:40.451411 27471 tablet_bootstrap.cc:492] T 15a380b9e3284d8381a0cef8ed33e826 P 29504c919f314b049deb56fffcff0261: Bootstrap starting.
I20260812 19:03:40.452812 27469 tablet_bootstrap.cc:654] T 15a380b9e3284d8381a0cef8ed33e826 P b63ba57646f740cc91c66d269496ec19: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:40.453282 27471 tablet_bootstrap.cc:654] T 15a380b9e3284d8381a0cef8ed33e826 P 29504c919f314b049deb56fffcff0261: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:40.453868 27473 tablet_bootstrap.cc:492] T 15a380b9e3284d8381a0cef8ed33e826 P 5bd3fc71996542bbbee59a45cd3f3200: Bootstrap starting.
I20260812 19:03:40.454010 27469 log.cc:824] T 15a380b9e3284d8381a0cef8ed33e826 P b63ba57646f740cc91c66d269496ec19: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:40.454145 27471 log.cc:824] T 15a380b9e3284d8381a0cef8ed33e826 P 29504c919f314b049deb56fffcff0261: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:40.456270 27469 tablet_bootstrap.cc:492] T 15a380b9e3284d8381a0cef8ed33e826 P b63ba57646f740cc91c66d269496ec19: No bootstrap required, opened a new log
I20260812 19:03:40.456472 27469 ts_tablet_manager.cc:1397] T 15a380b9e3284d8381a0cef8ed33e826 P b63ba57646f740cc91c66d269496ec19: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260812 19:03:40.456707 27471 tablet_bootstrap.cc:492] T 15a380b9e3284d8381a0cef8ed33e826 P 29504c919f314b049deb56fffcff0261: No bootstrap required, opened a new log
I20260812 19:03:40.456776 27473 tablet_bootstrap.cc:654] T 15a380b9e3284d8381a0cef8ed33e826 P 5bd3fc71996542bbbee59a45cd3f3200: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:40.456912 27471 ts_tablet_manager.cc:1397] T 15a380b9e3284d8381a0cef8ed33e826 P 29504c919f314b049deb56fffcff0261: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 19:03:40.457743 27473 log.cc:824] T 15a380b9e3284d8381a0cef8ed33e826 P 5bd3fc71996542bbbee59a45cd3f3200: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:40.459877 27469 raft_consensus.cc:348] T 15a380b9e3284d8381a0cef8ed33e826 P b63ba57646f740cc91c66d269496ec19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd3fc71996542bbbee59a45cd3f3200" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37611 } } peers { permanent_uuid: "b63ba57646f740cc91c66d269496ec19" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45925 } } peers { permanent_uuid: "29504c919f314b049deb56fffcff0261" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 43281 } }
I20260812 19:03:40.460132 27473 tablet_bootstrap.cc:492] T 15a380b9e3284d8381a0cef8ed33e826 P 5bd3fc71996542bbbee59a45cd3f3200: No bootstrap required, opened a new log
I20260812 19:03:40.460137 27469 raft_consensus.cc:374] T 15a380b9e3284d8381a0cef8ed33e826 P b63ba57646f740cc91c66d269496ec19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:40.460196 27469 raft_consensus.cc:729] T 15a380b9e3284d8381a0cef8ed33e826 P b63ba57646f740cc91c66d269496ec19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b63ba57646f740cc91c66d269496ec19, State: Initialized, Role: FOLLOWER
I20260812 19:03:40.460356 27473 ts_tablet_manager.cc:1397] T 15a380b9e3284d8381a0cef8ed33e826 P 5bd3fc71996542bbbee59a45cd3f3200: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 19:03:40.460654 27469 consensus_queue.cc:260] T 15a380b9e3284d8381a0cef8ed33e826 P b63ba57646f740cc91c66d269496ec19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd3fc71996542bbbee59a45cd3f3200" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37611 } } peers { permanent_uuid: "b63ba57646f740cc91c66d269496ec19" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45925 } } peers { permanent_uuid: "29504c919f314b049deb56fffcff0261" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 43281 } }
I20260812 19:03:40.461383 27449 heartbeater.cc:503] Master 127.25.130.62:42131 was elected leader, sending a full tablet report...
I20260812 19:03:40.461372 27471 raft_consensus.cc:348] T 15a380b9e3284d8381a0cef8ed33e826 P 29504c919f314b049deb56fffcff0261 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd3fc71996542bbbee59a45cd3f3200" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37611 } } peers { permanent_uuid: "b63ba57646f740cc91c66d269496ec19" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45925 } } peers { permanent_uuid: "29504c919f314b049deb56fffcff0261" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 43281 } }
I20260812 19:03:40.461697 27469 ts_tablet_manager.cc:1428] T 15a380b9e3284d8381a0cef8ed33e826 P b63ba57646f740cc91c66d269496ec19: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260812 19:03:40.461700 27471 raft_consensus.cc:374] T 15a380b9e3284d8381a0cef8ed33e826 P 29504c919f314b049deb56fffcff0261 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:40.461758 27471 raft_consensus.cc:729] T 15a380b9e3284d8381a0cef8ed33e826 P 29504c919f314b049deb56fffcff0261 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29504c919f314b049deb56fffcff0261, State: Initialized, Role: FOLLOWER
I20260812 19:03:40.462445 27471 consensus_queue.cc:260] T 15a380b9e3284d8381a0cef8ed33e826 P 29504c919f314b049deb56fffcff0261 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd3fc71996542bbbee59a45cd3f3200" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37611 } } peers { permanent_uuid: "b63ba57646f740cc91c66d269496ec19" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45925 } } peers { permanent_uuid: "29504c919f314b049deb56fffcff0261" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 43281 } }
I20260812 19:03:40.463186 27117 heartbeater.cc:503] Master 127.25.130.62:42131 was elected leader, sending a full tablet report...
I20260812 19:03:40.464905 27473 raft_consensus.cc:348] T 15a380b9e3284d8381a0cef8ed33e826 P 5bd3fc71996542bbbee59a45cd3f3200 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd3fc71996542bbbee59a45cd3f3200" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37611 } } peers { permanent_uuid: "b63ba57646f740cc91c66d269496ec19" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45925 } } peers { permanent_uuid: "29504c919f314b049deb56fffcff0261" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 43281 } }
I20260812 19:03:40.465226 27473 raft_consensus.cc:374] T 15a380b9e3284d8381a0cef8ed33e826 P 5bd3fc71996542bbbee59a45cd3f3200 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:40.465288 27473 raft_consensus.cc:729] T 15a380b9e3284d8381a0cef8ed33e826 P 5bd3fc71996542bbbee59a45cd3f3200 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bd3fc71996542bbbee59a45cd3f3200, State: Initialized, Role: FOLLOWER
I20260812 19:03:40.465380 27471 ts_tablet_manager.cc:1428] T 15a380b9e3284d8381a0cef8ed33e826 P 29504c919f314b049deb56fffcff0261: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260812 19:03:40.465780 27473 consensus_queue.cc:260] T 15a380b9e3284d8381a0cef8ed33e826 P 5bd3fc71996542bbbee59a45cd3f3200 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd3fc71996542bbbee59a45cd3f3200" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37611 } } peers { permanent_uuid: "b63ba57646f740cc91c66d269496ec19" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45925 } } peers { permanent_uuid: "29504c919f314b049deb56fffcff0261" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 43281 } }
I20260812 19:03:40.466504 27283 heartbeater.cc:503] Master 127.25.130.62:42131 was elected leader, sending a full tablet report...
I20260812 19:03:40.466898 27473 ts_tablet_manager.cc:1428] T 15a380b9e3284d8381a0cef8ed33e826 P 5bd3fc71996542bbbee59a45cd3f3200: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 19:03:40.570912 27482 raft_consensus.cc:482] T 15a380b9e3284d8381a0cef8ed33e826 P 29504c919f314b049deb56fffcff0261 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:03:40.571149 27482 raft_consensus.cc:504] T 15a380b9e3284d8381a0cef8ed33e826 P 29504c919f314b049deb56fffcff0261 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd3fc71996542bbbee59a45cd3f3200" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37611 } } peers { permanent_uuid: "b63ba57646f740cc91c66d269496ec19" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45925 } } peers { permanent_uuid: "29504c919f314b049deb56fffcff0261" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 43281 } }
I20260812 19:03:40.572374 27482 leader_election.cc:288] T 15a380b9e3284d8381a0cef8ed33e826 P 29504c919f314b049deb56fffcff0261 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5bd3fc71996542bbbee59a45cd3f3200 (127.25.130.2:37611), b63ba57646f740cc91c66d269496ec19 (127.25.130.3:45925)
I20260812 19:03:40.576475 27487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:40.572661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:37611 (local address 127.25.130.1:42111)
0812 19:03:40.573098 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:40.573110 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:40.573233 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:40.573489 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:40.573492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:40.573508 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:03:40.573675 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:40.573681 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.574412 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.574415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:40.575365 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:40.575381 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.576027 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.576030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:40.576138 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:40.576157 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 19:03:40.576224 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 19:03:40.576303 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":56,"thread_start_us":159,"threads_started":1}
I20260812 19:03:40.576454 27465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:40.572776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:42111 (local address 127.25.130.2:37611)
0812 19:03:40.572914 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:40.572919 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:40.573207 (+ 288us) server_negotiation.cc:407] Connection header received
0812 19:03:40.573341 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:40.573344 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:40.573390 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:40.573487 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:03:40.573785 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.574300 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.575499 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.575906 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.576017 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:40.576075 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:40.576252 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 19:03:40.576326 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 19:03:40.576519 27466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:40.572859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:43621 (local address 127.25.130.3:45925)
0812 19:03:40.572987 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:40.572992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:40.573292 (+ 300us) server_negotiation.cc:407] Connection header received
0812 19:03:40.573410 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:40.573413 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:40.573484 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:40.573551 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:03:40.573818 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.574305 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.575500 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.575906 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.576017 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:40.576127 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:40.576341 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 19:03:40.576417 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 19:03:40.576534 27485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:40.572714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:45925 (local address 127.25.130.1:43621)
0812 19:03:40.573169 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:40.573181 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:40.573308 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:40.573571 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:40.573574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:40.573590 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:03:40.573712 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:40.573717 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.574415 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.574418 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:40.575365 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:40.575381 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.576018 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.576021 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:40.576210 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:40.576232 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 19:03:40.576310 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 19:03:40.576380 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":34,"thread_start_us":133,"threads_started":1}
I20260812 19:03:40.577360 27397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15a380b9e3284d8381a0cef8ed33e826" candidate_uuid: "29504c919f314b049deb56fffcff0261" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b63ba57646f740cc91c66d269496ec19" is_pre_election: true
I20260812 19:03:40.577359 27225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15a380b9e3284d8381a0cef8ed33e826" candidate_uuid: "29504c919f314b049deb56fffcff0261" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bd3fc71996542bbbee59a45cd3f3200" is_pre_election: true
I20260812 19:03:40.577741 27225 raft_consensus.cc:2454] T 15a380b9e3284d8381a0cef8ed33e826 P 5bd3fc71996542bbbee59a45cd3f3200 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29504c919f314b049deb56fffcff0261 in term 0.
I20260812 19:03:40.577760 27397 raft_consensus.cc:2454] T 15a380b9e3284d8381a0cef8ed33e826 P b63ba57646f740cc91c66d269496ec19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29504c919f314b049deb56fffcff0261 in term 0.
I20260812 19:03:40.578315 26967 leader_election.cc:302] T 15a380b9e3284d8381a0cef8ed33e826 P 29504c919f314b049deb56fffcff0261 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29504c919f314b049deb56fffcff0261, 5bd3fc71996542bbbee59a45cd3f3200; no voters:
I20260812 19:03:40.578629 27482 raft_consensus.cc:2781] T 15a380b9e3284d8381a0cef8ed33e826 P 29504c919f314b049deb56fffcff0261 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:03:40.578733 27482 raft_consensus.cc:482] T 15a380b9e3284d8381a0cef8ed33e826 P 29504c919f314b049deb56fffcff0261 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:03:40.578785 27482 raft_consensus.cc:3037] T 15a380b9e3284d8381a0cef8ed33e826 P 29504c919f314b049deb56fffcff0261 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:40.579862 27482 raft_consensus.cc:504] T 15a380b9e3284d8381a0cef8ed33e826 P 29504c919f314b049deb56fffcff0261 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd3fc71996542bbbee59a45cd3f3200" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37611 } } peers { permanent_uuid: "b63ba57646f740cc91c66d269496ec19" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45925 } } peers { permanent_uuid: "29504c919f314b049deb56fffcff0261" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 43281 } }
I20260812 19:03:40.580216 27482 leader_election.cc:288] T 15a380b9e3284d8381a0cef8ed33e826 P 29504c919f314b049deb56fffcff0261 [CANDIDATE]: Term 1 election: Requested vote from peers 5bd3fc71996542bbbee59a45cd3f3200 (127.25.130.2:37611), b63ba57646f740cc91c66d269496ec19 (127.25.130.3:45925)
I20260812 19:03:40.580606 27225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15a380b9e3284d8381a0cef8ed33e826" candidate_uuid: "29504c919f314b049deb56fffcff0261" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bd3fc71996542bbbee59a45cd3f3200"
I20260812 19:03:40.580622 27397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15a380b9e3284d8381a0cef8ed33e826" candidate_uuid: "29504c919f314b049deb56fffcff0261" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b63ba57646f740cc91c66d269496ec19"
I20260812 19:03:40.580730 27225 raft_consensus.cc:3037] T 15a380b9e3284d8381a0cef8ed33e826 P 5bd3fc71996542bbbee59a45cd3f3200 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:40.580749 27397 raft_consensus.cc:3037] T 15a380b9e3284d8381a0cef8ed33e826 P b63ba57646f740cc91c66d269496ec19 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:40.581883 27225 raft_consensus.cc:2454] T 15a380b9e3284d8381a0cef8ed33e826 P 5bd3fc71996542bbbee59a45cd3f3200 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29504c919f314b049deb56fffcff0261 in term 1.
I20260812 19:03:40.581892 27397 raft_consensus.cc:2454] T 15a380b9e3284d8381a0cef8ed33e826 P b63ba57646f740cc91c66d269496ec19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29504c919f314b049deb56fffcff0261 in term 1.
I20260812 19:03:40.582197 26967 leader_election.cc:302] T 15a380b9e3284d8381a0cef8ed33e826 P 29504c919f314b049deb56fffcff0261 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29504c919f314b049deb56fffcff0261, 5bd3fc71996542bbbee59a45cd3f3200; no voters:
I20260812 19:03:40.582393 27482 raft_consensus.cc:2781] T 15a380b9e3284d8381a0cef8ed33e826 P 29504c919f314b049deb56fffcff0261 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:40.582635 27482 raft_consensus.cc:686] T 15a380b9e3284d8381a0cef8ed33e826 P 29504c919f314b049deb56fffcff0261 [term 1 LEADER]: Becoming Leader. State: Replica: 29504c919f314b049deb56fffcff0261, State: Running, Role: LEADER
I20260812 19:03:40.583016 27482 consensus_queue.cc:237] T 15a380b9e3284d8381a0cef8ed33e826 P 29504c919f314b049deb56fffcff0261 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd3fc71996542bbbee59a45cd3f3200" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37611 } } peers { permanent_uuid: "b63ba57646f740cc91c66d269496ec19" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45925 } } peers { permanent_uuid: "29504c919f314b049deb56fffcff0261" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 43281 } }
I20260812 19:03:40.586352 26861 catalog_manager.cc:5495] T 15a380b9e3284d8381a0cef8ed33e826 P 29504c919f314b049deb56fffcff0261 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29504c919f314b049deb56fffcff0261 (127.25.130.1). New cstate: current_term: 1 leader_uuid: "29504c919f314b049deb56fffcff0261" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bd3fc71996542bbbee59a45cd3f3200" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b63ba57646f740cc91c66d269496ec19" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29504c919f314b049deb56fffcff0261" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 43281 } health_report { overall_health: HEALTHY } } }
W20260812 19:03:40.629333 27450 tablet.cc:2369] T 15a380b9e3284d8381a0cef8ed33e826 P b63ba57646f740cc91c66d269496ec19: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 19:03:40 dist-test-slave-49h3 krb5kdc[26809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.1@KRBTEST.COM
I20260812 19:03:40.637210 27468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:40.624798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45570 (local address 127.25.130.1:43281)
0812 19:03:40.624958 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:40.624962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:40.625096 (+ 134us) server_negotiation.cc:407] Connection header received
0812 19:03:40.625215 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:40.625219 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:40.625287 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:40.625362 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:40.626157 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.626672 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.627541 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.627760 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.627853 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:40.633566 (+ 5713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:40.633582 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:40.633598 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:40.633631 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:40.635309 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:40.635805 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:40.635811 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:40.635813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:40.635904 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:40.636173 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:40.636176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:40.636178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:40.636460 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:40.636683 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:40.636886 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 19:03:40.637015 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260812 19:03:40.649047 27284 tablet.cc:2369] T 15a380b9e3284d8381a0cef8ed33e826 P 5bd3fc71996542bbbee59a45cd3f3200: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 19:03:40 dist-test-slave-49h3 krb5kdc[26809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561420, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 19:03:40 dist-test-slave-49h3 krb5kdc[26809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561420, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.130.1@KRBTEST.COM
I20260812 19:03:40.668319 27468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:40.659216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45576 (local address 127.25.130.1:43281)
0812 19:03:40.659428 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:40.659433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:40.659567 (+ 134us) server_negotiation.cc:407] Connection header received
0812 19:03:40.659722 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:40.659725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:40.659772 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:40.659844 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:40.660640 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.661193 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.662124 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.662372 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.662472 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:40.665090 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:40.665094 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:40.665096 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:40.665128 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:40.666569 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:40.667022 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:40.667025 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:40.667026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:40.667109 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:40.667383 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:40.667386 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:40.667388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:40.667579 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:40.667738 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:40.668023 (+ 285us) server_negotiation.cc:299] Negotiation successful
0812 19:03:40.668154 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260812 19:03:40.673580 27118 tablet.cc:2369] T 15a380b9e3284d8381a0cef8ed33e826 P 29504c919f314b049deb56fffcff0261: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 19:03:40 dist-test-slave-49h3 krb5kdc[26809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 19:03:40 dist-test-slave-49h3 krb5kdc[26809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.1@KRBTEST.COM
I20260812 19:03:40.696441 27468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:40.687898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45584 (local address 127.25.130.1:43281)
0812 19:03:40.688086 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:40.688091 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:40.688236 (+ 145us) server_negotiation.cc:407] Connection header received
0812 19:03:40.688386 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:40.688389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:40.688437 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:40.688506 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:40.689254 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.689842 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.690701 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:40.690925 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:40.691015 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:40.693609 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:40.693612 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:40.693614 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:40.693645 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:40.694933 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:40.695392 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:40.695395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:40.695397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:40.695482 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:40.695684 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:40.695686 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:40.695687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:40.695856 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:40.695991 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:40.696199 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 19:03:40.696311 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260812 19:03:40.697929 26999 tablet_service.cc:2908] Rejecting scan request for tablet 15a380b9e3284d8381a0cef8ed33e826: Uninitialized: clean time has not yet been initialized
I20260812 19:03:40.700201 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 26943
I20260812 19:03:40.711097 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 27122
I20260812 19:03:40.720642 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 27291
I20260812 19:03:40.731024 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 26823
2026-08-12T19:03:40Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2451 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:03:40 dist-test-slave-49h3 krb5kdc[27506](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:03:40 dist-test-slave-49h3 krb5kdc[27506](info): set up 2 sockets
Aug 12 19:03:40 dist-test-slave-49h3 krb5kdc[27506](info): commencing operation
krb5kdc: starting...
W20260812 19:03:41.806811 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:03:41 dist-test-slave-49h3 krb5kdc[27506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:03:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:03:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:03:41.937738 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:38489
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:42965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:38489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:42.055686 27521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:42.055986 27521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:42.056043 27521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:42.060204 27521 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:42.060290 27521 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 19:03:42.060314 27521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:42.060338 27521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:42.060360 27521 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:42.060385 27521 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:42.066116 27521 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:42965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:38489
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:38489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:42.067304 27521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:42.068425 27521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:42.077291 27528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:42.077323 27529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:42.077301 27531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:42.077389 27521 server_base.cc:1034] running on GCE node
I20260812 19:03:42.078204 27521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:42.079384 27521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:42.080549 27521 hybrid_clock.cc:648] HybridClock initialized: now 1786561422080521 us; error 45 us; skew 500 ppm
Aug 12 19:03:42 dist-test-slave-49h3 krb5kdc[27506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561422, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:42.083406 27521 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:03:42.084388 27521 webserver.cc:460] Webserver started at http://127.25.130.62:41895/ using document root <none> and password file <none>
I20260812 19:03:42.084949 27521 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:42.085008 27521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:42.085198 27521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:42.087414 27521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "b7278dd442364a00b761d7cd3d3467ae"
format_stamp: "Formatted at 2026-08-12 19:03:42 on dist-test-slave-49h3"
server_key: "7402ef52e92e7ed61cf1668dbaff1fb9"
server_key_iv: "66728967305e916f63f8cdaedf499326"
server_key_version: "encryptionkey@0"
I20260812 19:03:42.087965 27521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "b7278dd442364a00b761d7cd3d3467ae"
format_stamp: "Formatted at 2026-08-12 19:03:42 on dist-test-slave-49h3"
server_key: "7402ef52e92e7ed61cf1668dbaff1fb9"
server_key_iv: "66728967305e916f63f8cdaedf499326"
server_key_version: "encryptionkey@0"
I20260812 19:03:42.091944 27521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:03:42.094558 27538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:42.095856 27521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:03:42.095990 27521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "b7278dd442364a00b761d7cd3d3467ae"
format_stamp: "Formatted at 2026-08-12 19:03:42 on dist-test-slave-49h3"
server_key: "7402ef52e92e7ed61cf1668dbaff1fb9"
server_key_iv: "66728967305e916f63f8cdaedf499326"
server_key_version: "encryptionkey@0"
I20260812 19:03:42.096122 27521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:42.122239 27521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:42.125312 27521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:42.125540 27521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:42.134102 27521 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:38489
I20260812 19:03:42.134131 27622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:38489 every 8 connection(s)
I20260812 19:03:42.135128 27521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:42.138073 27624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:42.143164 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 27521
I20260812 19:03:42.143277 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:03:42.143497 26120 external_mini_cluster.cc:1428] Setting key 5e28c578c30454fc36db4ca790d53593
I20260812 19:03:42.144001 27624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7278dd442364a00b761d7cd3d3467ae: Bootstrap starting.
I20260812 19:03:42.146914 27624 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7278dd442364a00b761d7cd3d3467ae: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:42.147902 27624 log.cc:824] T 00000000000000000000000000000000 P b7278dd442364a00b761d7cd3d3467ae: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:42.150058 27624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7278dd442364a00b761d7cd3d3467ae: No bootstrap required, opened a new log
Aug 12 19:03:42 dist-test-slave-49h3 krb5kdc[27506](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:42.154358 27624 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7278dd442364a00b761d7cd3d3467ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7278dd442364a00b761d7cd3d3467ae" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 38489 } }
I20260812 19:03:42.154696 27624 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7278dd442364a00b761d7cd3d3467ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:42.154762 27624 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7278dd442364a00b761d7cd3d3467ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7278dd442364a00b761d7cd3d3467ae, State: Initialized, Role: FOLLOWER
I20260812 19:03:42.155228 27624 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7278dd442364a00b761d7cd3d3467ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7278dd442364a00b761d7cd3d3467ae" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 38489 } }
I20260812 19:03:42.155382 27624 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7278dd442364a00b761d7cd3d3467ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:03:42.155452 27624 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7278dd442364a00b761d7cd3d3467ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:03:42.155560 27624 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7278dd442364a00b761d7cd3d3467ae [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:42.156564 27624 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7278dd442364a00b761d7cd3d3467ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7278dd442364a00b761d7cd3d3467ae" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 38489 } }
I20260812 19:03:42.156927 27624 leader_election.cc:302] T 00000000000000000000000000000000 P b7278dd442364a00b761d7cd3d3467ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7278dd442364a00b761d7cd3d3467ae; no voters:
I20260812 19:03:42.157286 27624 leader_election.cc:288] T 00000000000000000000000000000000 P b7278dd442364a00b761d7cd3d3467ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:03:42.157378 27629 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7278dd442364a00b761d7cd3d3467ae [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:42.157635 27629 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7278dd442364a00b761d7cd3d3467ae [term 1 LEADER]: Becoming Leader. State: Replica: b7278dd442364a00b761d7cd3d3467ae, State: Running, Role: LEADER
I20260812 19:03:42.157971 27629 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7278dd442364a00b761d7cd3d3467ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7278dd442364a00b761d7cd3d3467ae" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 38489 } }
I20260812 19:03:42.158759 27624 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7278dd442364a00b761d7cd3d3467ae [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:42.158916 27627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:42.145011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60656 (local address 127.25.130.62:38489)
0812 19:03:42.145536 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:42.145557 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:42.145602 (+ 45us) server_negotiation.cc:407] Connection header received
0812 19:03:42.146535 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:42.146562 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:42.147051 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:42.147474 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:42.148205 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.149245 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.150201 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.150928 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.151104 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:42.153603 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:42.153611 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:42.153632 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:42.153681 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:42.155638 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:42.156114 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:42.156121 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:42.156129 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:42.156288 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:42.156576 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:42.156580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:42.156582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:42.157086 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:42.157306 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:42.157680 (+ 374us) server_negotiation.cc:299] Negotiation successful
0812 19:03:42.158010 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":158,"threads_started":1}
I20260812 19:03:42.159317 27629 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7278dd442364a00b761d7cd3d3467ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7278dd442364a00b761d7cd3d3467ae. Latest consensus state: current_term: 1 leader_uuid: "b7278dd442364a00b761d7cd3d3467ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7278dd442364a00b761d7cd3d3467ae" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 38489 } } }
I20260812 19:03:42.159437 27629 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7278dd442364a00b761d7cd3d3467ae [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:42.159842 27635 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:42.159945 27630 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7278dd442364a00b761d7cd3d3467ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7278dd442364a00b761d7cd3d3467ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7278dd442364a00b761d7cd3d3467ae" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 38489 } } }
I20260812 19:03:42.160111 27630 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7278dd442364a00b761d7cd3d3467ae [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:42.163357 27635 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:03:42.168790 27635 catalog_manager.cc:1347] Generated new cluster ID: 570cd23c96744a45bfedef369db55407
I20260812 19:03:42.168877 27635 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:42.183954 27635 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:42.185346 27635 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:42.198199 27635 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7278dd442364a00b761d7cd3d3467ae: Generated new TSK 0
I20260812 19:03:42.199141 27635 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
I20260812 19:03:42.230875 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:0
--local_ip_for_outbound_sockets=127.25.130.1
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:38489
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:42965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:42.349577 27657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:42.349884 27657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:42.349931 27657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:42.354182 27657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:42.354269 27657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:42.354310 27657 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 19:03:42.354348 27657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:03:42.360301 27657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:42965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.130.62:38489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:42.361584 27657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:42.362704 27657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:42.373519 27664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:42.373519 27663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:42.373577 27666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:42.373643 27657 server_base.cc:1034] running on GCE node
I20260812 19:03:42.374308 27657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:42.375046 27657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:42.376262 27657 hybrid_clock.cc:648] HybridClock initialized: now 1786561422376242 us; error 37 us; skew 500 ppm
Aug 12 19:03:42 dist-test-slave-49h3 krb5kdc[27506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561422, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:42.379619 27657 init.cc:377] Logged in from keytab as kudu/127.25.130.1@KRBTEST.COM (short username kudu)
I20260812 19:03:42.380656 27657 webserver.cc:460] Webserver started at http://127.25.130.1:44539/ using document root <none> and password file <none>
I20260812 19:03:42.381251 27657 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:42.381325 27657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:42.381579 27657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:42.383809 27657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-0/data/instance:
uuid: "a997ea4b0a7b49e8b95c8a2456c62c61"
format_stamp: "Formatted at 2026-08-12 19:03:42 on dist-test-slave-49h3"
server_key: "7a068a37186d7ec83815c3d1c12b4449"
server_key_iv: "50da8b11d15a5485d17cd25ca7e3e595"
server_key_version: "encryptionkey@0"
I20260812 19:03:42.384413 27657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance:
uuid: "a997ea4b0a7b49e8b95c8a2456c62c61"
format_stamp: "Formatted at 2026-08-12 19:03:42 on dist-test-slave-49h3"
server_key: "7a068a37186d7ec83815c3d1c12b4449"
server_key_iv: "50da8b11d15a5485d17cd25ca7e3e595"
server_key_version: "encryptionkey@0"
I20260812 19:03:42.388638 27657 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 19:03:42.391438 27677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:42.392871 27657 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 19:03:42.393024 27657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "a997ea4b0a7b49e8b95c8a2456c62c61"
format_stamp: "Formatted at 2026-08-12 19:03:42 on dist-test-slave-49h3"
server_key: "7a068a37186d7ec83815c3d1c12b4449"
server_key_iv: "50da8b11d15a5485d17cd25ca7e3e595"
server_key_version: "encryptionkey@0"
I20260812 19:03:42.393157 27657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:42.410003 27657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:42.413288 27657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:42.413527 27657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:42.415158 27657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:03:42.415215 27657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:42.415273 27657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:03:42.415302 27657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 19:03:42 dist-test-slave-49h3 krb5kdc[27506](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561422, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:42.426285 27657 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:42525
I20260812 19:03:42.426450 27829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:42525 every 8 connection(s)
I20260812 19:03:42.427320 27657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:03:42.429107 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 27657
I20260812 19:03:42.429215 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance
I20260812 19:03:42.429513 26120 external_mini_cluster.cc:1428] Setting key 502ca01d324754e2123fe9fbeb016e63
I20260812 19:03:42.430094 27627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:42.416177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:33163 (local address 127.25.130.62:38489)
0812 19:03:42.416344 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:42.416350 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:42.417250 (+ 900us) server_negotiation.cc:407] Connection header received
0812 19:03:42.418443 (+ 1193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:42.418447 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:42.418512 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:42.418593 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:42.420009 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.420539 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.422419 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.422772 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.422884 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:42.426115 (+ 3231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:42.426120 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:42.426124 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:42.426162 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:42.427681 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:42.428300 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:42.428303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:42.428305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:42.428399 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:42.428785 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:42.428789 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:42.428791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:42.429007 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:42.429105 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:42.429750 (+ 645us) server_negotiation.cc:299] Negotiation successful
0812 19:03:42.429909 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 19:03:42.431070 27688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:42.416563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:38489 (local address 127.25.130.1:33163)
0812 19:03:42.417054 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:42.417121 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:42.418218 (+ 1097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:42.418798 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:42.418808 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:42.419191 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:42.419843 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:42.419860 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.420689 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.420694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:42.422253 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:42.422271 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.422944 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.422949 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:42.423179 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:42.423862 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:42.423886 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:42.425840 (+ 1954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:42.427816 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:42.427824 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:42.427841 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:42.428176 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:42.428500 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:42.428504 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:42.428506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:42.428667 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:42.429121 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:42.429129 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:42.429438 (+ 309us) client_negotiation.cc:769] Sending connection context
0812 19:03:42.429686 (+ 248us) client_negotiation.cc:240] Negotiation successful
0812 19:03:42.430060 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":113,"threads_started":1}
I20260812 19:03:42.432577 27830 heartbeater.cc:347] Connected to a master server at 127.25.130.62:38489
I20260812 19:03:42.432754 27830 heartbeater.cc:464] Registering TS with master...
I20260812 19:03:42.433418 27830 heartbeater.cc:511] Master 127.25.130.62:38489 requested a full tablet report, sending...
I20260812 19:03:42.435420 27558 ts_manager.cc:194] Registered new tserver with Master: a997ea4b0a7b49e8b95c8a2456c62c61 (127.25.130.1:42525)
I20260812 19:03:42.437062 27558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.1@KRBTEST.COM'} at 127.25.130.1:33163
WARNING: no policy specified for kudu/127.25.130.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.2@KRBTEST.COM" created.
I20260812 19:03:42.445086 27627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:42.435137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:37129 (local address 127.25.130.62:38489)
0812 19:03:42.435274 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:42.435279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:42.437776 (+ 2497us) server_negotiation.cc:407] Connection header received
0812 19:03:42.437927 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:42.437931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:42.437982 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:42.438034 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:42.438627 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.439157 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.439987 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.440290 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.440394 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:42.441760 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:42.441764 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:42.441766 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:42.441793 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:42.443229 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:42.443792 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:42.443795 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:42.443796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:42.443885 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:42.444165 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:42.444168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:42.444170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:42.444377 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:42.444470 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:42.444779 (+ 309us) server_negotiation.cc:299] Negotiation successful
0812 19:03:42.444902 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 19:03:42.445101 27838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:42.435055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:38489 (local address 127.25.130.1:37129)
0812 19:03:42.437674 (+ 2619us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:42.437688 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:42.437809 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:42.438186 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:42.438189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:42.438394 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:42.438507 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:42.438512 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.439289 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.439292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:42.439860 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:42.439870 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.440438 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.440442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:42.440553 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:42.441084 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:42.441100 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:42.441593 (+ 493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:42.443372 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:42.443375 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:42.443377 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:42.443692 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:42.443963 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:42.443965 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:42.443967 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:42.444054 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:42.444484 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:42.444487 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:42.444607 (+ 120us) client_negotiation.cc:769] Sending connection context
0812 19:03:42.444705 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 19:03:42.444897 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2512,"thread_start_us":101,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
I20260812 19:03:42.460621 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:0
--local_ip_for_outbound_sockets=127.25.130.2
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:38489
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:42965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:42.579557 27840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:42.579860 27840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:42.579906 27840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:42.584102 27840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:42.584192 27840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:42.584241 27840 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 19:03:42.584275 27840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:03:42.589952 27840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:42965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.130.62:38489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:42.591079 27840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:42.592111 27840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:42.602555 27854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:42.602555 27848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:42.602614 27840 server_base.cc:1034] running on GCE node
W20260812 19:03:42.602555 27851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:42.603302 27840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:42.603962 27840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:42.605159 27840 hybrid_clock.cc:648] HybridClock initialized: now 1786561422605143 us; error 42 us; skew 500 ppm
Aug 12 19:03:42 dist-test-slave-49h3 krb5kdc[27506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561422, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:42.608078 27840 init.cc:377] Logged in from keytab as kudu/127.25.130.2@KRBTEST.COM (short username kudu)
I20260812 19:03:42.609059 27840 webserver.cc:460] Webserver started at http://127.25.130.2:39063/ using document root <none> and password file <none>
I20260812 19:03:42.609686 27840 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:42.609762 27840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:42.609984 27840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:42.612151 27840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-1/data/instance:
uuid: "38e8f436073b4f5494a52113d186a0c1"
format_stamp: "Formatted at 2026-08-12 19:03:42 on dist-test-slave-49h3"
server_key: "514c4dd51dbfdc6bc32f683444a2f666"
server_key_iv: "6955730d6a046e4b23b43a767adabcdf"
server_key_version: "encryptionkey@0"
I20260812 19:03:42.612728 27840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance:
uuid: "38e8f436073b4f5494a52113d186a0c1"
format_stamp: "Formatted at 2026-08-12 19:03:42 on dist-test-slave-49h3"
server_key: "514c4dd51dbfdc6bc32f683444a2f666"
server_key_iv: "6955730d6a046e4b23b43a767adabcdf"
server_key_version: "encryptionkey@0"
I20260812 19:03:42.616693 27840 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 19:03:42.619330 27863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:42.620595 27840 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:03:42.620726 27840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "38e8f436073b4f5494a52113d186a0c1"
format_stamp: "Formatted at 2026-08-12 19:03:42 on dist-test-slave-49h3"
server_key: "514c4dd51dbfdc6bc32f683444a2f666"
server_key_iv: "6955730d6a046e4b23b43a767adabcdf"
server_key_version: "encryptionkey@0"
I20260812 19:03:42.620839 27840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:42.643563 27840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:42.646611 27840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:42.646816 27840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:42.648412 27840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:03:42.648469 27840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:42.648525 27840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:03:42.648554 27840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 19:03:42 dist-test-slave-49h3 krb5kdc[27506](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561422, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:42.659534 27840 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:46273
I20260812 19:03:42.659570 28011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:46273 every 8 connection(s)
I20260812 19:03:42.661170 27840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:03:42.663102 27627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:42.649267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:52663 (local address 127.25.130.62:38489)
0812 19:03:42.649410 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:42.649415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:42.650424 (+ 1009us) server_negotiation.cc:407] Connection header received
0812 19:03:42.651676 (+ 1252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:42.651680 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:42.651744 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:42.651839 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:42.653173 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.653843 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.655027 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.655281 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.655388 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:42.658326 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:42.658330 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:42.658333 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:42.658369 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:42.660146 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:42.661556 (+ 1410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:42.661561 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:42.661562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:42.661672 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:42.662044 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:42.662047 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:42.662049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:42.662291 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:42.662395 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:42.662859 (+ 464us) server_negotiation.cc:299] Negotiation successful
0812 19:03:42.662975 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 19:03:42.663889 27873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:42.649737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:38489 (local address 127.25.130.2:52663)
0812 19:03:42.650240 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:42.650303 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:42.651424 (+ 1121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:42.652026 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:42.652034 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:42.652454 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:42.653009 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:42.653024 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.653972 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.653975 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:42.654894 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:42.654904 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.655444 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.655448 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:42.655617 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:42.656211 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:42.656229 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:42.658119 (+ 1890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:42.660938 (+ 2819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:42.660989 (+ 51us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:42.661047 (+ 58us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:42.661422 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:42.661772 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:42.661775 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:42.661777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:42.661944 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:42.662383 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:42.662388 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:42.662629 (+ 241us) client_negotiation.cc:769] Sending connection context
0812 19:03:42.662788 (+ 159us) client_negotiation.cc:240] Negotiation successful
0812 19:03:42.663059 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":111,"threads_started":1}
I20260812 19:03:42.665100 28012 heartbeater.cc:347] Connected to a master server at 127.25.130.62:38489
I20260812 19:03:42.665256 28012 heartbeater.cc:464] Registering TS with master...
I20260812 19:03:42.665812 28012 heartbeater.cc:511] Master 127.25.130.62:38489 requested a full tablet report, sending...
I20260812 19:03:42.666875 27560 ts_manager.cc:194] Registered new tserver with Master: 38e8f436073b4f5494a52113d186a0c1 (127.25.130.2:46273)
I20260812 19:03:42.667529 27560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.2@KRBTEST.COM'} at 127.25.130.2:52663
I20260812 19:03:42.669014 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 27840
I20260812 19:03:42.669113 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance
I20260812 19:03:42.669409 26120 external_mini_cluster.cc:1428] Setting key 7b6667ff3795f641e905421e6e88dc4c
I20260812 19:03:42.674706 27627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:42.667468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:41785 (local address 127.25.130.62:38489)
0812 19:03:42.667589 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:42.667594 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:42.667729 (+ 135us) server_negotiation.cc:407] Connection header received
0812 19:03:42.667864 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:42.667867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:42.667915 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:42.667985 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:42.668451 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.669208 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.670190 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.670428 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.670505 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:42.671658 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:42.671661 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:42.671663 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:42.671687 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:42.672997 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:42.673403 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:42.673406 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:42.673408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:42.673528 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:42.673764 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:42.673766 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:42.673768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:42.674073 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:42.674200 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:42.674466 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 19:03:42.674575 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":51}
I20260812 19:03:42.674749 28019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:42.667359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:38489 (local address 127.25.130.2:41785)
0812 19:03:42.667657 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:42.667673 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:42.667777 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:42.668016 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:42.668019 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:42.668227 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:42.668343 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:42.668350 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.669342 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.669346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:42.670080 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:42.670091 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.670538 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.670542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:42.670627 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:42.671083 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:42.671097 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:42.671524 (+ 427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:42.673118 (+ 1594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:42.673121 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:42.673122 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:42.673323 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:42.673607 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:42.673609 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:42.673610 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:42.673689 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:42.674189 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:42.674192 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:42.674300 (+ 108us) client_negotiation.cc:769] Sending connection context
0812 19:03:42.674413 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 19:03:42.674566 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":1,"thread_start_us":105,"threads_started":1}
WARNING: no policy specified for kudu/127.25.130.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
I20260812 19:03:42.698552 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:0
--local_ip_for_outbound_sockets=127.25.130.3
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:38489
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:42965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:42.814750 28022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:42.815058 28022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:42.815124 28022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:42.819306 28022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:42.819393 28022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:42.819449 28022 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 19:03:42.819507 28022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:03:42.825284 28022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:42965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.130.62:38489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:42.826489 28022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:42.827565 28022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:42.837775 28032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:42.837803 28029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:42.837842 28027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:42.837903 28022 server_base.cc:1034] running on GCE node
I20260812 19:03:42.838546 28022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:42.839314 28022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:42.840492 28022 hybrid_clock.cc:648] HybridClock initialized: now 1786561422840471 us; error 36 us; skew 500 ppm
Aug 12 19:03:42 dist-test-slave-49h3 krb5kdc[27506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561422, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:42.843461 28022 init.cc:377] Logged in from keytab as kudu/127.25.130.3@KRBTEST.COM (short username kudu)
I20260812 19:03:42.844506 28022 webserver.cc:460] Webserver started at http://127.25.130.3:43565/ using document root <none> and password file <none>
I20260812 19:03:42.845191 28022 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:42.845273 28022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:42.845583 28022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:42.847922 28022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-2/data/instance:
uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec"
format_stamp: "Formatted at 2026-08-12 19:03:42 on dist-test-slave-49h3"
server_key: "398f67e9db5b6f2d5fa7d2d636ab3f65"
server_key_iv: "4bcfe4e4577a0020bd32cb1c277020d6"
server_key_version: "encryptionkey@0"
I20260812 19:03:42.848646 28022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance:
uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec"
format_stamp: "Formatted at 2026-08-12 19:03:42 on dist-test-slave-49h3"
server_key: "398f67e9db5b6f2d5fa7d2d636ab3f65"
server_key_iv: "4bcfe4e4577a0020bd32cb1c277020d6"
server_key_version: "encryptionkey@0"
I20260812 19:03:42.853103 28022 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 19:03:42.855995 28040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:42.857324 28022 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 19:03:42.857488 28022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec"
format_stamp: "Formatted at 2026-08-12 19:03:42 on dist-test-slave-49h3"
server_key: "398f67e9db5b6f2d5fa7d2d636ab3f65"
server_key_iv: "4bcfe4e4577a0020bd32cb1c277020d6"
server_key_version: "encryptionkey@0"
I20260812 19:03:42.857614 28022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:42.891155 28022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:42.894310 28022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:42.894517 28022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:42.896065 28022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:03:42.896121 28022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:42.896179 28022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:03:42.896209 28022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 19:03:42 dist-test-slave-49h3 krb5kdc[27506](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561422, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:42.907056 28022 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:38817
I20260812 19:03:42.907121 28178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:38817 every 8 connection(s)
I20260812 19:03:42.908082 28022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:03:42.910369 27627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:42.897021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:49321 (local address 127.25.130.62:38489)
0812 19:03:42.897151 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:42.897154 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:42.898348 (+ 1194us) server_negotiation.cc:407] Connection header received
0812 19:03:42.899617 (+ 1269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:42.899621 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:42.899687 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:42.899761 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:42.901214 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.901844 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.903296 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.903533 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.903635 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:42.906573 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:42.906578 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:42.906580 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:42.906617 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:42.908201 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:42.908870 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:42.908874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:42.908876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:42.908986 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:42.909308 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:42.909311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:42.909312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:42.909571 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:42.909652 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:42.910121 (+ 469us) server_negotiation.cc:299] Negotiation successful
0812 19:03:42.910245 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 19:03:42.911167 28052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:42.897648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:38489 (local address 127.25.130.3:49321)
0812 19:03:42.898175 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:42.898223 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:42.899365 (+ 1142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:42.899952 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:42.899959 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:42.900431 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:42.901055 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:42.901070 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.901976 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.901980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:42.903163 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:42.903176 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.903703 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.903707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:42.903896 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:42.904491 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:42.904508 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:42.906296 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:42.908350 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:42.908356 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:42.908374 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:42.908740 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:42.909077 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:42.909080 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:42.909081 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:42.909226 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:42.909662 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:42.909668 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:42.909907 (+ 239us) client_negotiation.cc:769] Sending connection context
0812 19:03:42.910068 (+ 161us) client_negotiation.cc:240] Negotiation successful
0812 19:03:42.910335 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":147,"threads_started":1}
I20260812 19:03:42.912540 28179 heartbeater.cc:347] Connected to a master server at 127.25.130.62:38489
I20260812 19:03:42.912705 28179 heartbeater.cc:464] Registering TS with master...
I20260812 19:03:42.913249 28179 heartbeater.cc:511] Master 127.25.130.62:38489 requested a full tablet report, sending...
I20260812 19:03:42.914489 27558 ts_manager.cc:194] Registered new tserver with Master: f4f41b8e5c3141279dd8ff3b1e7e74ec (127.25.130.3:38817)
I20260812 19:03:42.915169 27558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.3@KRBTEST.COM'} at 127.25.130.3:49321
I20260812 19:03:42.916910 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 28022
I20260812 19:03:42.917042 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance
I20260812 19:03:42.917330 26120 external_mini_cluster.cc:1428] Setting key 13a54dc3f1714507758df8fc1c81154f
I20260812 19:03:42.919458 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:03:42.922266 27627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:42.914721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:59973 (local address 127.25.130.62:38489)
0812 19:03:42.914853 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:42.914858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:42.915079 (+ 221us) server_negotiation.cc:407] Connection header received
0812 19:03:42.915178 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:42.915181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:42.915231 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:42.915308 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:42.915928 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.916691 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.917827 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.918070 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.918163 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:42.919305 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:42.919309 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:42.919312 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:42.919339 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:42.920606 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:42.921027 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:42.921032 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:42.921034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:42.921116 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:42.921381 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:42.921384 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:42.921386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:42.921636 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:42.921735 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:42.921989 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 19:03:42.922104 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 19:03:42.922286 28188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:42.914618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:38489 (local address 127.25.130.3:59973)
0812 19:03:42.914996 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:42.915008 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:42.915106 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:42.915428 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:42.915431 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:42.915674 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:42.915809 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:42.915815 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.916839 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.916843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:42.917712 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:42.917722 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.918177 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.918181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:42.918272 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:42.918820 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:42.918833 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:42.919169 (+ 336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:42.920721 (+ 1552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:42.920724 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:42.920726 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:42.920934 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:42.921206 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:42.921209 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:42.921215 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:42.921299 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:42.921735 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:42.921739 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:42.921837 (+ 98us) client_negotiation.cc:769] Sending connection context
0812 19:03:42.921925 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 19:03:42.922094 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":1,"thread_start_us":112,"threads_started":1}
Aug 12 19:03:42 dist-test-slave-49h3 krb5kdc[27506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561422, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 19:03:42 dist-test-slave-49h3 krb5kdc[27506](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561422, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:42.947250 27627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:42.939280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60666 (local address 127.25.130.62:38489)
0812 19:03:42.939416 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:42.939420 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:42.939433 (+ 13us) server_negotiation.cc:407] Connection header received
0812 19:03:42.939476 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:42.939479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:42.939522 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:42.939591 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:42.940345 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.940890 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.941785 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.942007 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.942085 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:42.944411 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:42.944415 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:42.944417 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:42.944449 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:42.945810 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:42.946241 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:42.946243 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:42.946244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:42.946319 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:42.946521 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:42.946523 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:42.946524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:42.946719 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:42.946814 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:42.947013 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 19:03:42.947117 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 19:03:42.950453 27558 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60666:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:03:42.953603 27558 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:03:42.968941 28206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:42.963053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42856 (local address 127.25.130.1:42525)
0812 19:03:42.963773 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:42.963779 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:42.963802 (+ 23us) server_negotiation.cc:407] Connection header received
0812 19:03:42.963884 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:42.963889 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:42.964112 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:42.964266 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:03:42.964648 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.965776 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.967388 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.968168 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.968436 (+ 268us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:42.968629 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:42.968684 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 19:03:42.968800 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":86,"threads_started":1}
I20260812 19:03:42.968931 28201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:42.962899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:42525 (local address 127.0.0.1:42856)
0812 19:03:42.963325 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:42.963382 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:42.963492 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:42.964280 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:42.964286 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:42.964330 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:03:42.964486 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:42.964494 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.965885 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.965888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:42.967248 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:42.967259 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.968318 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.968322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:42.968474 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:42.968532 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 19:03:42.968633 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 19:03:42.968762 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":20,"thread_start_us":95,"threads_started":1}
I20260812 19:03:42.968942 28207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:42.962953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:46273 (local address 127.0.0.1:49808)
0812 19:03:42.963919 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:42.963932 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:42.964013 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:42.964691 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:42.964694 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:42.964713 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:03:42.965065 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:42.965074 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.966503 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.966507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:42.967507 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:42.967518 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.968594 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.968598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:42.968705 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:42.968727 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 19:03:42.968791 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 19:03:42.968849 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":905,"mutex_wait_us":5,"thread_start_us":90,"threads_started":1}
I20260812 19:03:42.969024 28208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:42.963736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49808 (local address 127.25.130.2:46273)
0812 19:03:42.964265 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:42.964277 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:42.964299 (+ 22us) server_negotiation.cc:407] Connection header received
0812 19:03:42.964368 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:42.964373 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:42.964534 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:42.965229 (+ 695us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:03:42.965345 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.966408 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.967640 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.968448 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.968588 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:42.968732 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:42.968817 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 19:03:42.968898 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":89,"threads_started":1}
I20260812 19:03:42.971889 28202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:42.963670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:38817 (local address 127.0.0.1:39610)
0812 19:03:42.965977 (+ 2307us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:42.965991 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:42.966101 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:42.966693 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:42.966696 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:42.966714 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:03:42.966817 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:42.966822 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.968106 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.968109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:42.970126 (+ 2017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:42.970138 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.971429 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.971433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:42.971535 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:42.971564 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 19:03:42.971642 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 19:03:42.971722 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2247,"mutex_wait_us":1,"thread_start_us":102,"threads_started":1}
I20260812 19:03:42.972087 27746 tablet_service.cc:1467] Processing CreateTablet for tablet 6b5f3b734ab04dd4b3421311d55b5c70 (DEFAULT_TABLE table=test-table [id=e147728872a74af7bda79853f526b132]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:03:42.972510 28209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:42.964063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39610 (local address 127.25.130.3:38817)
0812 19:03:42.965839 (+ 1776us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:42.965851 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:42.966102 (+ 251us) server_negotiation.cc:407] Connection header received
0812 19:03:42.966205 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:42.966210 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:42.966387 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:42.966535 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:03:42.966943 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.967987 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.970272 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:42.971062 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:42.971176 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:42.971347 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:42.972254 (+ 907us) server_negotiation.cc:299] Negotiation successful
0812 19:03:42.972370 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1674,"thread_start_us":114,"threads_started":1}
I20260812 19:03:42.972175 27927 tablet_service.cc:1467] Processing CreateTablet for tablet 6b5f3b734ab04dd4b3421311d55b5c70 (DEFAULT_TABLE table=test-table [id=e147728872a74af7bda79853f526b132]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:03:42.973266 27927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b5f3b734ab04dd4b3421311d55b5c70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:42.973265 27746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b5f3b734ab04dd4b3421311d55b5c70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:42.975806 28102 tablet_service.cc:1467] Processing CreateTablet for tablet 6b5f3b734ab04dd4b3421311d55b5c70 (DEFAULT_TABLE table=test-table [id=e147728872a74af7bda79853f526b132]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:03:42.977113 28102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b5f3b734ab04dd4b3421311d55b5c70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:42.980054 28210 tablet_bootstrap.cc:492] T 6b5f3b734ab04dd4b3421311d55b5c70 P a997ea4b0a7b49e8b95c8a2456c62c61: Bootstrap starting.
I20260812 19:03:42.981971 28211 tablet_bootstrap.cc:492] T 6b5f3b734ab04dd4b3421311d55b5c70 P 38e8f436073b4f5494a52113d186a0c1: Bootstrap starting.
I20260812 19:03:42.983062 28210 tablet_bootstrap.cc:654] T 6b5f3b734ab04dd4b3421311d55b5c70 P a997ea4b0a7b49e8b95c8a2456c62c61: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:42.984207 28210 log.cc:824] T 6b5f3b734ab04dd4b3421311d55b5c70 P a997ea4b0a7b49e8b95c8a2456c62c61: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:42.984452 28211 tablet_bootstrap.cc:654] T 6b5f3b734ab04dd4b3421311d55b5c70 P 38e8f436073b4f5494a52113d186a0c1: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:42.985338 28211 log.cc:824] T 6b5f3b734ab04dd4b3421311d55b5c70 P 38e8f436073b4f5494a52113d186a0c1: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:42.985502 28213 tablet_bootstrap.cc:492] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec: Bootstrap starting.
I20260812 19:03:42.986630 28210 tablet_bootstrap.cc:492] T 6b5f3b734ab04dd4b3421311d55b5c70 P a997ea4b0a7b49e8b95c8a2456c62c61: No bootstrap required, opened a new log
I20260812 19:03:42.986845 28210 ts_tablet_manager.cc:1397] T 6b5f3b734ab04dd4b3421311d55b5c70 P a997ea4b0a7b49e8b95c8a2456c62c61: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 19:03:42.987561 28211 tablet_bootstrap.cc:492] T 6b5f3b734ab04dd4b3421311d55b5c70 P 38e8f436073b4f5494a52113d186a0c1: No bootstrap required, opened a new log
I20260812 19:03:42.987775 28211 ts_tablet_manager.cc:1397] T 6b5f3b734ab04dd4b3421311d55b5c70 P 38e8f436073b4f5494a52113d186a0c1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 19:03:42.987859 28213 tablet_bootstrap.cc:654] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:42.988756 28213 log.cc:824] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:42.991164 28213 tablet_bootstrap.cc:492] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec: No bootstrap required, opened a new log
I20260812 19:03:42.991389 28213 ts_tablet_manager.cc:1397] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 19:03:42.991278 28211 raft_consensus.cc:348] T 6b5f3b734ab04dd4b3421311d55b5c70 P 38e8f436073b4f5494a52113d186a0c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } }
I20260812 19:03:42.991299 28210 raft_consensus.cc:348] T 6b5f3b734ab04dd4b3421311d55b5c70 P a997ea4b0a7b49e8b95c8a2456c62c61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } }
I20260812 19:03:42.991551 28211 raft_consensus.cc:374] T 6b5f3b734ab04dd4b3421311d55b5c70 P 38e8f436073b4f5494a52113d186a0c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:42.991580 28210 raft_consensus.cc:374] T 6b5f3b734ab04dd4b3421311d55b5c70 P a997ea4b0a7b49e8b95c8a2456c62c61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:42.991623 28211 raft_consensus.cc:729] T 6b5f3b734ab04dd4b3421311d55b5c70 P 38e8f436073b4f5494a52113d186a0c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38e8f436073b4f5494a52113d186a0c1, State: Initialized, Role: FOLLOWER
I20260812 19:03:42.991683 28210 raft_consensus.cc:729] T 6b5f3b734ab04dd4b3421311d55b5c70 P a997ea4b0a7b49e8b95c8a2456c62c61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a997ea4b0a7b49e8b95c8a2456c62c61, State: Initialized, Role: FOLLOWER
I20260812 19:03:42.992090 28211 consensus_queue.cc:260] T 6b5f3b734ab04dd4b3421311d55b5c70 P 38e8f436073b4f5494a52113d186a0c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } }
I20260812 19:03:42.992163 28210 consensus_queue.cc:260] T 6b5f3b734ab04dd4b3421311d55b5c70 P a997ea4b0a7b49e8b95c8a2456c62c61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } }
I20260812 19:03:42.992779 28012 heartbeater.cc:503] Master 127.25.130.62:38489 was elected leader, sending a full tablet report...
I20260812 19:03:42.993167 28211 ts_tablet_manager.cc:1428] T 6b5f3b734ab04dd4b3421311d55b5c70 P 38e8f436073b4f5494a52113d186a0c1: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 19:03:42.993170 28210 ts_tablet_manager.cc:1428] T 6b5f3b734ab04dd4b3421311d55b5c70 P a997ea4b0a7b49e8b95c8a2456c62c61: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 19:03:42.993570 27830 heartbeater.cc:503] Master 127.25.130.62:38489 was elected leader, sending a full tablet report...
I20260812 19:03:42.995290 28213 raft_consensus.cc:348] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } }
I20260812 19:03:42.995633 28213 raft_consensus.cc:374] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:42.995736 28213 raft_consensus.cc:729] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4f41b8e5c3141279dd8ff3b1e7e74ec, State: Initialized, Role: FOLLOWER
I20260812 19:03:42.996225 28213 consensus_queue.cc:260] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } }
I20260812 19:03:42.997404 28213 ts_tablet_manager.cc:1428] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 19:03:42.997536 28179 heartbeater.cc:503] Master 127.25.130.62:38489 was elected leader, sending a full tablet report...
I20260812 19:03:43.009649 28218 raft_consensus.cc:482] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:03:43.009869 28218 raft_consensus.cc:504] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } }
I20260812 19:03:43.011011 28218 leader_election.cc:288] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a997ea4b0a7b49e8b95c8a2456c62c61 (127.25.130.1:42525), 38e8f436073b4f5494a52113d186a0c1 (127.25.130.2:46273)
I20260812 19:03:43.014806 28208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:43.011433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:39863 (local address 127.25.130.2:46273)
0812 19:03:43.011590 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:43.011593 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:43.011607 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:03:43.011658 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:43.011660 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:43.011699 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:43.011780 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:03:43.012056 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:43.012637 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:43.013851 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:43.014257 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:43.014370 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:43.014428 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:43.014582 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 19:03:43.014664 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 19:03:43.014806 28052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:43.011308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:46273 (local address 127.25.130.3:39863)
0812 19:03:43.011417 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:43.011432 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:43.011534 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:43.011814 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:43.011817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:43.011838 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:03:43.011946 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:43.011952 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:43.012752 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:43.012756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:43.013725 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:43.013736 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:43.014371 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:43.014374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:43.014474 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:43.014492 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 19:03:43.014556 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 19:03:43.014678 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":22}
I20260812 19:03:43.015529 27955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b5f3b734ab04dd4b3421311d55b5c70" candidate_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38e8f436073b4f5494a52113d186a0c1" is_pre_election: true
I20260812 19:03:43.015925 27955 raft_consensus.cc:2454] T 6b5f3b734ab04dd4b3421311d55b5c70 P 38e8f436073b4f5494a52113d186a0c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4f41b8e5c3141279dd8ff3b1e7e74ec in term 0.
I20260812 19:03:43.016044 28206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:43.011433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:37413 (local address 127.25.130.1:42525)
0812 19:03:43.011588 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:43.011592 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:43.012059 (+ 467us) server_negotiation.cc:407] Connection header received
0812 19:03:43.012141 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:43.012144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:43.012190 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:43.012251 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:03:43.012530 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:43.013036 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:43.014814 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:43.015423 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:43.015546 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:43.015591 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:43.015822 (+ 231us) server_negotiation.cc:299] Negotiation successful
0812 19:03:43.015915 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 19:03:43.016425 28219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:43.011308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:42525 (local address 127.25.130.3:37413)
0812 19:03:43.011944 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:43.011955 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:43.012059 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:43.012266 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:43.012269 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:43.012285 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:03:43.012433 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:43.012440 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:43.013139 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:43.013143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:43.014684 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:43.014695 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:43.015568 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:43.015572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:43.015666 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:43.015692 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 19:03:43.016196 (+ 504us) client_negotiation.cc:240] Negotiation successful
0812 19:03:43.016282 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"mutex_wait_us":297,"thread_start_us":130,"threads_started":1}
I20260812 19:03:43.016556 28044 leader_election.cc:302] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38e8f436073b4f5494a52113d186a0c1, f4f41b8e5c3141279dd8ff3b1e7e74ec; no voters:
I20260812 19:03:43.016886 28218 raft_consensus.cc:2781] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:03:43.016999 28218 raft_consensus.cc:482] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:03:43.017048 28218 raft_consensus.cc:3037] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:43.017093 27771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b5f3b734ab04dd4b3421311d55b5c70" candidate_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" is_pre_election: true
I20260812 19:03:43.017472 27771 raft_consensus.cc:2454] T 6b5f3b734ab04dd4b3421311d55b5c70 P a997ea4b0a7b49e8b95c8a2456c62c61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4f41b8e5c3141279dd8ff3b1e7e74ec in term 0.
I20260812 19:03:43.018311 28218 raft_consensus.cc:504] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } }
I20260812 19:03:43.018638 28218 leader_election.cc:288] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec [CANDIDATE]: Term 1 election: Requested vote from peers a997ea4b0a7b49e8b95c8a2456c62c61 (127.25.130.1:42525), 38e8f436073b4f5494a52113d186a0c1 (127.25.130.2:46273)
I20260812 19:03:43.019011 27771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b5f3b734ab04dd4b3421311d55b5c70" candidate_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61"
I20260812 19:03:43.019029 27955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b5f3b734ab04dd4b3421311d55b5c70" candidate_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38e8f436073b4f5494a52113d186a0c1"
I20260812 19:03:43.019133 27771 raft_consensus.cc:3037] T 6b5f3b734ab04dd4b3421311d55b5c70 P a997ea4b0a7b49e8b95c8a2456c62c61 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:43.019219 27955 raft_consensus.cc:3037] T 6b5f3b734ab04dd4b3421311d55b5c70 P 38e8f436073b4f5494a52113d186a0c1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:43.020223 27771 raft_consensus.cc:2454] T 6b5f3b734ab04dd4b3421311d55b5c70 P a997ea4b0a7b49e8b95c8a2456c62c61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4f41b8e5c3141279dd8ff3b1e7e74ec in term 1.
I20260812 19:03:43.020541 28041 leader_election.cc:302] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a997ea4b0a7b49e8b95c8a2456c62c61, f4f41b8e5c3141279dd8ff3b1e7e74ec; no voters:
I20260812 19:03:43.020700 28218 raft_consensus.cc:2781] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:43.020784 27955 raft_consensus.cc:2454] T 6b5f3b734ab04dd4b3421311d55b5c70 P 38e8f436073b4f5494a52113d186a0c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4f41b8e5c3141279dd8ff3b1e7e74ec in term 1.
I20260812 19:03:43.020921 28218 raft_consensus.cc:686] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 1 LEADER]: Becoming Leader. State: Replica: f4f41b8e5c3141279dd8ff3b1e7e74ec, State: Running, Role: LEADER
I20260812 19:03:43.021256 28218 consensus_queue.cc:237] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } }
I20260812 19:03:43.024518 27558 catalog_manager.cc:5495] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec reported cstate change: term changed from 0 to 1, leader changed from <none> to f4f41b8e5c3141279dd8ff3b1e7e74ec (127.25.130.3). New cstate: current_term: 1 leader_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } health_report { overall_health: HEALTHY } } }
I20260812 19:03:43.043237 28209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:43.039557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39618 (local address 127.25.130.3:38817)
0812 19:03:43.039706 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:43.039711 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:43.039727 (+ 16us) server_negotiation.cc:407] Connection header received
0812 19:03:43.039923 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:43.039925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:43.039984 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:43.040071 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:03:43.040311 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:43.040819 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:43.041735 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:43.041963 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:43.042088 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:43.042391 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:03:43.042848 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:03:43.042939 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:43.042999 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 19:03:43.043077 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 19:03:43.049301 27955 raft_consensus.cc:1261] T 6b5f3b734ab04dd4b3421311d55b5c70 P 38e8f436073b4f5494a52113d186a0c1 [term 1 FOLLOWER]: Refusing update from remote peer f4f41b8e5c3141279dd8ff3b1e7e74ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:03:43.049301 27771 raft_consensus.cc:1261] T 6b5f3b734ab04dd4b3421311d55b5c70 P a997ea4b0a7b49e8b95c8a2456c62c61 [term 1 FOLLOWER]: Refusing update from remote peer f4f41b8e5c3141279dd8ff3b1e7e74ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:03:43.050050 28220 consensus_queue.cc:1035] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:03:43.050239 28218 consensus_queue.cc:1035] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:03:43.072306 27560 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:03:43.072486 27560 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:03:43.075138 27560 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7278dd442364a00b761d7cd3d3467ae: Sending DeleteTablet for 3 replicas of tablet 6b5f3b734ab04dd4b3421311d55b5c70
I20260812 19:03:43.076025 28102 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b5f3b734ab04dd4b3421311d55b5c70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:03:43 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:39610
I20260812 19:03:43.076074 27927 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b5f3b734ab04dd4b3421311d55b5c70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:03:43 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:49808
I20260812 19:03:43.076165 27746 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b5f3b734ab04dd4b3421311d55b5c70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:03:43 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:42856
I20260812 19:03:43.076625 28241 tablet_replica.cc:330] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec: stopping tablet replica
I20260812 19:03:43.076961 28241 raft_consensus.cc:2229] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:03:43.076977 28242 tablet_replica.cc:330] T 6b5f3b734ab04dd4b3421311d55b5c70 P 38e8f436073b4f5494a52113d186a0c1: stopping tablet replica
I20260812 19:03:43.077512 28241 raft_consensus.cc:2258] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:03:43.077512 28242 raft_consensus.cc:2229] T 6b5f3b734ab04dd4b3421311d55b5c70 P 38e8f436073b4f5494a52113d186a0c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:03:43.077857 28242 raft_consensus.cc:2258] T 6b5f3b734ab04dd4b3421311d55b5c70 P 38e8f436073b4f5494a52113d186a0c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:03:43.077785 27560 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60666:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:03:43.078250 27560 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:03:43.078320 28243 tablet_replica.cc:330] T 6b5f3b734ab04dd4b3421311d55b5c70 P a997ea4b0a7b49e8b95c8a2456c62c61: stopping tablet replica
I20260812 19:03:43.078639 28243 raft_consensus.cc:2229] T 6b5f3b734ab04dd4b3421311d55b5c70 P a997ea4b0a7b49e8b95c8a2456c62c61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:03:43.079279 28241 ts_tablet_manager.cc:1905] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:03:43.079490 28242 ts_tablet_manager.cc:1905] T 6b5f3b734ab04dd4b3421311d55b5c70 P 38e8f436073b4f5494a52113d186a0c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:03:43.078931 28243 raft_consensus.cc:2258] T 6b5f3b734ab04dd4b3421311d55b5c70 P a997ea4b0a7b49e8b95c8a2456c62c61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:03:43.080957 28243 ts_tablet_manager.cc:1905] T 6b5f3b734ab04dd4b3421311d55b5c70 P a997ea4b0a7b49e8b95c8a2456c62c61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:03:43.082507 28242 ts_tablet_manager.cc:1918] T 6b5f3b734ab04dd4b3421311d55b5c70 P 38e8f436073b4f5494a52113d186a0c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:03:43.083455 28241 ts_tablet_manager.cc:1918] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:03:43.083645 28241 log.cc:1196] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-2/wal/wals/6b5f3b734ab04dd4b3421311d55b5c70
I20260812 19:03:43.084160 27746 tablet_service.cc:1467] Processing CreateTablet for tablet 91f4ae1567d84b48b36931bebf59650d (DEFAULT_TABLE table=test-table [id=98fcfcc48bd94690a5e0eeaf8b54a778]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:03:43.084159 28102 tablet_service.cc:1467] Processing CreateTablet for tablet 91f4ae1567d84b48b36931bebf59650d (DEFAULT_TABLE table=test-table [id=98fcfcc48bd94690a5e0eeaf8b54a778]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:03:43.084451 27746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91f4ae1567d84b48b36931bebf59650d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:43.084514 28102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91f4ae1567d84b48b36931bebf59650d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:43.084801 28243 ts_tablet_manager.cc:1918] T 6b5f3b734ab04dd4b3421311d55b5c70 P a997ea4b0a7b49e8b95c8a2456c62c61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:03:43.084903 28243 log.cc:1196] T 6b5f3b734ab04dd4b3421311d55b5c70 P a997ea4b0a7b49e8b95c8a2456c62c61: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-0/wal/wals/6b5f3b734ab04dd4b3421311d55b5c70
I20260812 19:03:43.085222 28243 ts_tablet_manager.cc:1939] T 6b5f3b734ab04dd4b3421311d55b5c70 P a997ea4b0a7b49e8b95c8a2456c62c61: Deleting consensus metadata
I20260812 19:03:43.085577 28242 log.cc:1196] T 6b5f3b734ab04dd4b3421311d55b5c70 P 38e8f436073b4f5494a52113d186a0c1: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-1/wal/wals/6b5f3b734ab04dd4b3421311d55b5c70
I20260812 19:03:43.085922 28242 ts_tablet_manager.cc:1939] T 6b5f3b734ab04dd4b3421311d55b5c70 P 38e8f436073b4f5494a52113d186a0c1: Deleting consensus metadata
I20260812 19:03:43.086704 28213 tablet_bootstrap.cc:492] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec: Bootstrap starting.
I20260812 19:03:43.086760 28210 tablet_bootstrap.cc:492] T 91f4ae1567d84b48b36931bebf59650d P a997ea4b0a7b49e8b95c8a2456c62c61: Bootstrap starting.
I20260812 19:03:43.086817 27540 catalog_manager.cc:4841] TS a997ea4b0a7b49e8b95c8a2456c62c61 (127.25.130.1:42525): tablet 6b5f3b734ab04dd4b3421311d55b5c70 (table test-table [id=e147728872a74af7bda79853f526b132]) successfully deleted
I20260812 19:03:43.087426 28241 ts_tablet_manager.cc:1939] T 6b5f3b734ab04dd4b3421311d55b5c70 P f4f41b8e5c3141279dd8ff3b1e7e74ec: Deleting consensus metadata
I20260812 19:03:43.087721 28213 tablet_bootstrap.cc:654] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:43.087724 28210 tablet_bootstrap.cc:654] T 91f4ae1567d84b48b36931bebf59650d P a997ea4b0a7b49e8b95c8a2456c62c61: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:43.088624 27540 catalog_manager.cc:4841] TS f4f41b8e5c3141279dd8ff3b1e7e74ec (127.25.130.3:38817): tablet 6b5f3b734ab04dd4b3421311d55b5c70 (table test-table [id=e147728872a74af7bda79853f526b132]) successfully deleted
I20260812 19:03:43.088941 27927 tablet_service.cc:1467] Processing CreateTablet for tablet 91f4ae1567d84b48b36931bebf59650d (DEFAULT_TABLE table=test-table [id=98fcfcc48bd94690a5e0eeaf8b54a778]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:03:43.088959 27543 catalog_manager.cc:4841] TS 38e8f436073b4f5494a52113d186a0c1 (127.25.130.2:46273): tablet 6b5f3b734ab04dd4b3421311d55b5c70 (table test-table [id=e147728872a74af7bda79853f526b132]) successfully deleted
I20260812 19:03:43.089222 27927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91f4ae1567d84b48b36931bebf59650d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:43.089759 28210 tablet_bootstrap.cc:492] T 91f4ae1567d84b48b36931bebf59650d P a997ea4b0a7b49e8b95c8a2456c62c61: No bootstrap required, opened a new log
I20260812 19:03:43.089834 28210 ts_tablet_manager.cc:1397] T 91f4ae1567d84b48b36931bebf59650d P a997ea4b0a7b49e8b95c8a2456c62c61: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 19:03:43.090772 28210 raft_consensus.cc:348] T 91f4ae1567d84b48b36931bebf59650d P a997ea4b0a7b49e8b95c8a2456c62c61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } }
I20260812 19:03:43.090886 28210 raft_consensus.cc:374] T 91f4ae1567d84b48b36931bebf59650d P a997ea4b0a7b49e8b95c8a2456c62c61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:43.090917 28210 raft_consensus.cc:729] T 91f4ae1567d84b48b36931bebf59650d P a997ea4b0a7b49e8b95c8a2456c62c61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a997ea4b0a7b49e8b95c8a2456c62c61, State: Initialized, Role: FOLLOWER
I20260812 19:03:43.091068 28210 consensus_queue.cc:260] T 91f4ae1567d84b48b36931bebf59650d P a997ea4b0a7b49e8b95c8a2456c62c61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } }
I20260812 19:03:43.091178 28213 tablet_bootstrap.cc:492] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec: No bootstrap required, opened a new log
I20260812 19:03:43.091259 28213 ts_tablet_manager.cc:1397] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 19:03:43.091713 28213 raft_consensus.cc:348] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } }
I20260812 19:03:43.091828 28213 raft_consensus.cc:374] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:43.091851 28211 tablet_bootstrap.cc:492] T 91f4ae1567d84b48b36931bebf59650d P 38e8f436073b4f5494a52113d186a0c1: Bootstrap starting.
I20260812 19:03:43.091858 28213 raft_consensus.cc:729] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4f41b8e5c3141279dd8ff3b1e7e74ec, State: Initialized, Role: FOLLOWER
I20260812 19:03:43.091984 28213 consensus_queue.cc:260] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } }
I20260812 19:03:43.092264 28213 ts_tablet_manager.cc:1428] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 19:03:43.092871 28211 tablet_bootstrap.cc:654] T 91f4ae1567d84b48b36931bebf59650d P 38e8f436073b4f5494a52113d186a0c1: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:43.093202 28210 ts_tablet_manager.cc:1428] T 91f4ae1567d84b48b36931bebf59650d P a997ea4b0a7b49e8b95c8a2456c62c61: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 19:03:43.094154 28211 tablet_bootstrap.cc:492] T 91f4ae1567d84b48b36931bebf59650d P 38e8f436073b4f5494a52113d186a0c1: No bootstrap required, opened a new log
I20260812 19:03:43.094269 28211 ts_tablet_manager.cc:1397] T 91f4ae1567d84b48b36931bebf59650d P 38e8f436073b4f5494a52113d186a0c1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 19:03:43.094730 28211 raft_consensus.cc:348] T 91f4ae1567d84b48b36931bebf59650d P 38e8f436073b4f5494a52113d186a0c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } }
I20260812 19:03:43.094851 28211 raft_consensus.cc:374] T 91f4ae1567d84b48b36931bebf59650d P 38e8f436073b4f5494a52113d186a0c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:43.094883 28211 raft_consensus.cc:729] T 91f4ae1567d84b48b36931bebf59650d P 38e8f436073b4f5494a52113d186a0c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38e8f436073b4f5494a52113d186a0c1, State: Initialized, Role: FOLLOWER
I20260812 19:03:43.094998 28211 consensus_queue.cc:260] T 91f4ae1567d84b48b36931bebf59650d P 38e8f436073b4f5494a52113d186a0c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } }
I20260812 19:03:43.095263 28211 ts_tablet_manager.cc:1428] T 91f4ae1567d84b48b36931bebf59650d P 38e8f436073b4f5494a52113d186a0c1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 19:03:43.148814 28235 raft_consensus.cc:482] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:03:43.148967 28235 raft_consensus.cc:504] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } }
I20260812 19:03:43.149474 28235 leader_election.cc:288] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a997ea4b0a7b49e8b95c8a2456c62c61 (127.25.130.1:42525), 38e8f436073b4f5494a52113d186a0c1 (127.25.130.2:46273)
I20260812 19:03:43.149959 27771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91f4ae1567d84b48b36931bebf59650d" candidate_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" is_pre_election: true
I20260812 19:03:43.150139 27771 raft_consensus.cc:2454] T 91f4ae1567d84b48b36931bebf59650d P a997ea4b0a7b49e8b95c8a2456c62c61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4f41b8e5c3141279dd8ff3b1e7e74ec in term 0.
I20260812 19:03:43.150100 27955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91f4ae1567d84b48b36931bebf59650d" candidate_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38e8f436073b4f5494a52113d186a0c1" is_pre_election: true
I20260812 19:03:43.150264 27955 raft_consensus.cc:2454] T 91f4ae1567d84b48b36931bebf59650d P 38e8f436073b4f5494a52113d186a0c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4f41b8e5c3141279dd8ff3b1e7e74ec in term 0.
I20260812 19:03:43.150470 28041 leader_election.cc:302] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a997ea4b0a7b49e8b95c8a2456c62c61, f4f41b8e5c3141279dd8ff3b1e7e74ec; no voters:
I20260812 19:03:43.150714 28235 raft_consensus.cc:2781] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:03:43.150771 28235 raft_consensus.cc:482] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:03:43.150813 28235 raft_consensus.cc:3037] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:43.151841 28235 raft_consensus.cc:504] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } }
I20260812 19:03:43.152171 28235 leader_election.cc:288] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec [CANDIDATE]: Term 1 election: Requested vote from peers a997ea4b0a7b49e8b95c8a2456c62c61 (127.25.130.1:42525), 38e8f436073b4f5494a52113d186a0c1 (127.25.130.2:46273)
I20260812 19:03:43.152554 27771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91f4ae1567d84b48b36931bebf59650d" candidate_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61"
I20260812 19:03:43.152635 27955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91f4ae1567d84b48b36931bebf59650d" candidate_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38e8f436073b4f5494a52113d186a0c1"
I20260812 19:03:43.152702 27771 raft_consensus.cc:3037] T 91f4ae1567d84b48b36931bebf59650d P a997ea4b0a7b49e8b95c8a2456c62c61 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:43.152755 27955 raft_consensus.cc:3037] T 91f4ae1567d84b48b36931bebf59650d P 38e8f436073b4f5494a52113d186a0c1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:43.153895 27955 raft_consensus.cc:2454] T 91f4ae1567d84b48b36931bebf59650d P 38e8f436073b4f5494a52113d186a0c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4f41b8e5c3141279dd8ff3b1e7e74ec in term 1.
I20260812 19:03:43.153894 27771 raft_consensus.cc:2454] T 91f4ae1567d84b48b36931bebf59650d P a997ea4b0a7b49e8b95c8a2456c62c61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4f41b8e5c3141279dd8ff3b1e7e74ec in term 1.
I20260812 19:03:43.154260 28041 leader_election.cc:302] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 38e8f436073b4f5494a52113d186a0c1, a997ea4b0a7b49e8b95c8a2456c62c61, f4f41b8e5c3141279dd8ff3b1e7e74ec; no voters:
I20260812 19:03:43.154448 28235 raft_consensus.cc:2781] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:43.154531 28235 raft_consensus.cc:686] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 1 LEADER]: Becoming Leader. State: Replica: f4f41b8e5c3141279dd8ff3b1e7e74ec, State: Running, Role: LEADER
I20260812 19:03:43.154665 28235 consensus_queue.cc:237] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } }
I20260812 19:03:43.156301 27558 catalog_manager.cc:5495] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec reported cstate change: term changed from 0 to 1, leader changed from <none> to f4f41b8e5c3141279dd8ff3b1e7e74ec (127.25.130.3). New cstate: current_term: 1 leader_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } health_report { overall_health: HEALTHY } } }
W20260812 19:03:43.159013 28180 tablet.cc:2369] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 19:03:43.162477 28013 tablet.cc:2369] T 91f4ae1567d84b48b36931bebf59650d P 38e8f436073b4f5494a52113d186a0c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 19:03:43.179303 27831 tablet.cc:2369] T 91f4ae1567d84b48b36931bebf59650d P a997ea4b0a7b49e8b95c8a2456c62c61: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 19:03:43 dist-test-slave-49h3 krb5kdc[27506](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561422, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:43.216302 27627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:43.207485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60678 (local address 127.25.130.62:38489)
0812 19:03:43.207602 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:43.207606 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:43.207847 (+ 241us) server_negotiation.cc:407] Connection header received
0812 19:03:43.207972 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:43.207975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:43.208015 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:43.208088 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:43.208805 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:43.209325 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:43.210192 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:43.210420 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:43.210512 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:43.212899 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:43.212902 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:43.212905 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:43.212960 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:43.214504 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:43.214994 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:43.214998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:43.215000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:43.215081 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:43.215322 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:43.215325 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:43.215327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:43.215541 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:43.215631 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:43.215959 (+ 328us) server_negotiation.cc:299] Negotiation successful
0812 19:03:43.216165 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":101,"server-negotiator.queue_time_us":40}
I20260812 19:03:43.216305 28256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:43.207454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:38489 (local address 127.0.0.1:60678)
0812 19:03:43.207755 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:43.207767 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:43.207888 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:43.208124 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:43.208126 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:43.208569 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:43.208678 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:43.208686 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:43.209427 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:43.209431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:43.210071 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:43.210083 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:43.210525 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:43.210529 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:43.210612 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:43.211183 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:43.211196 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:43.212753 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:43.214626 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:43.214639 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:43.214646 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:43.214898 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:43.215161 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:43.215164 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:43.215166 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:43.215246 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:43.215645 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:43.215653 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:43.215780 (+ 127us) client_negotiation.cc:769] Sending connection context
0812 19:03:43.215894 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 19:03:43.216096 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":142,"threads_started":1}
I20260812 19:03:43.221791 27558 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:60678:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 19:03:43.222662 27558 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:03:43.228672 27927 tablet_service.cc:1467] Processing CreateTablet for tablet 14463a5cfa924dbd8493979192a30284 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5fdf4ff5fb7948e1b31fa0682abb781c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 19:03:43.228673 27746 tablet_service.cc:1467] Processing CreateTablet for tablet 14463a5cfa924dbd8493979192a30284 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5fdf4ff5fb7948e1b31fa0682abb781c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 19:03:43.229156 27746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14463a5cfa924dbd8493979192a30284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:43.229156 27927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14463a5cfa924dbd8493979192a30284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:43.232333 28211 tablet_bootstrap.cc:492] T 14463a5cfa924dbd8493979192a30284 P 38e8f436073b4f5494a52113d186a0c1: Bootstrap starting.
I20260812 19:03:43.232791 28102 tablet_service.cc:1467] Processing CreateTablet for tablet 14463a5cfa924dbd8493979192a30284 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5fdf4ff5fb7948e1b31fa0682abb781c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 19:03:43.233220 28102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14463a5cfa924dbd8493979192a30284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:43.233625 28211 tablet_bootstrap.cc:654] T 14463a5cfa924dbd8493979192a30284 P 38e8f436073b4f5494a52113d186a0c1: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:43.235209 28211 tablet_bootstrap.cc:492] T 14463a5cfa924dbd8493979192a30284 P 38e8f436073b4f5494a52113d186a0c1: No bootstrap required, opened a new log
I20260812 19:03:43.235304 28211 ts_tablet_manager.cc:1397] T 14463a5cfa924dbd8493979192a30284 P 38e8f436073b4f5494a52113d186a0c1: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 19:03:43.235772 28211 raft_consensus.cc:348] T 14463a5cfa924dbd8493979192a30284 P 38e8f436073b4f5494a52113d186a0c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } }
I20260812 19:03:43.235890 28211 raft_consensus.cc:374] T 14463a5cfa924dbd8493979192a30284 P 38e8f436073b4f5494a52113d186a0c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:43.235917 28211 raft_consensus.cc:729] T 14463a5cfa924dbd8493979192a30284 P 38e8f436073b4f5494a52113d186a0c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38e8f436073b4f5494a52113d186a0c1, State: Initialized, Role: FOLLOWER
I20260812 19:03:43.235955 28213 tablet_bootstrap.cc:492] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec: Bootstrap starting.
I20260812 19:03:43.236037 28211 consensus_queue.cc:260] T 14463a5cfa924dbd8493979192a30284 P 38e8f436073b4f5494a52113d186a0c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } }
I20260812 19:03:43.236334 28211 ts_tablet_manager.cc:1428] T 14463a5cfa924dbd8493979192a30284 P 38e8f436073b4f5494a52113d186a0c1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 19:03:43.236440 28216 tablet_replica.cc:439] T 14463a5cfa924dbd8493979192a30284 P 38e8f436073b4f5494a52113d186a0c1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } } }
I20260812 19:03:43.236629 28216 tablet_replica.cc:442] T 14463a5cfa924dbd8493979192a30284 P 38e8f436073b4f5494a52113d186a0c1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 19:03:43.236984 28213 tablet_bootstrap.cc:654] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:43.238467 28210 tablet_bootstrap.cc:492] T 14463a5cfa924dbd8493979192a30284 P a997ea4b0a7b49e8b95c8a2456c62c61: Bootstrap starting.
I20260812 19:03:43.239416 28210 tablet_bootstrap.cc:654] T 14463a5cfa924dbd8493979192a30284 P a997ea4b0a7b49e8b95c8a2456c62c61: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:43.240752 28210 tablet_bootstrap.cc:492] T 14463a5cfa924dbd8493979192a30284 P a997ea4b0a7b49e8b95c8a2456c62c61: No bootstrap required, opened a new log
I20260812 19:03:43.240836 28210 ts_tablet_manager.cc:1397] T 14463a5cfa924dbd8493979192a30284 P a997ea4b0a7b49e8b95c8a2456c62c61: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 19:03:43.240991 28213 tablet_bootstrap.cc:492] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec: No bootstrap required, opened a new log
I20260812 19:03:43.241083 28213 ts_tablet_manager.cc:1397] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260812 19:03:43.241318 28210 raft_consensus.cc:348] T 14463a5cfa924dbd8493979192a30284 P a997ea4b0a7b49e8b95c8a2456c62c61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } }
I20260812 19:03:43.241482 28210 raft_consensus.cc:374] T 14463a5cfa924dbd8493979192a30284 P a997ea4b0a7b49e8b95c8a2456c62c61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:43.241524 28210 raft_consensus.cc:729] T 14463a5cfa924dbd8493979192a30284 P a997ea4b0a7b49e8b95c8a2456c62c61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a997ea4b0a7b49e8b95c8a2456c62c61, State: Initialized, Role: FOLLOWER
I20260812 19:03:43.241683 28210 consensus_queue.cc:260] T 14463a5cfa924dbd8493979192a30284 P a997ea4b0a7b49e8b95c8a2456c62c61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } }
I20260812 19:03:43.241787 28213 raft_consensus.cc:348] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } }
I20260812 19:03:43.241917 28213 raft_consensus.cc:374] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:43.241957 28213 raft_consensus.cc:729] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4f41b8e5c3141279dd8ff3b1e7e74ec, State: Initialized, Role: FOLLOWER
I20260812 19:03:43.241943 28217 tablet_replica.cc:439] T 14463a5cfa924dbd8493979192a30284 P a997ea4b0a7b49e8b95c8a2456c62c61: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } } }
I20260812 19:03:43.242117 28217 tablet_replica.cc:442] T 14463a5cfa924dbd8493979192a30284 P a997ea4b0a7b49e8b95c8a2456c62c61: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 19:03:43.242096 28213 consensus_queue.cc:260] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } }
I20260812 19:03:43.242375 28213 ts_tablet_manager.cc:1428] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 19:03:43.242326 28235 tablet_replica.cc:439] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } } }
I20260812 19:03:43.242473 28235 tablet_replica.cc:442] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 19:03:43.242806 28210 ts_tablet_manager.cc:1428] T 14463a5cfa924dbd8493979192a30284 P a997ea4b0a7b49e8b95c8a2456c62c61: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 19:03:43.448204 28235 raft_consensus.cc:482] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:03:43.448364 28235 raft_consensus.cc:504] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } }
I20260812 19:03:43.448792 28235 leader_election.cc:288] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a997ea4b0a7b49e8b95c8a2456c62c61 (127.25.130.1:42525), 38e8f436073b4f5494a52113d186a0c1 (127.25.130.2:46273)
I20260812 19:03:43.449288 27771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14463a5cfa924dbd8493979192a30284" candidate_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" is_pre_election: true
I20260812 19:03:43.449326 27955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14463a5cfa924dbd8493979192a30284" candidate_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38e8f436073b4f5494a52113d186a0c1" is_pre_election: true
I20260812 19:03:43.449492 27955 raft_consensus.cc:2454] T 14463a5cfa924dbd8493979192a30284 P 38e8f436073b4f5494a52113d186a0c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4f41b8e5c3141279dd8ff3b1e7e74ec in term 0.
I20260812 19:03:43.449497 27771 raft_consensus.cc:2454] T 14463a5cfa924dbd8493979192a30284 P a997ea4b0a7b49e8b95c8a2456c62c61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4f41b8e5c3141279dd8ff3b1e7e74ec in term 0.
I20260812 19:03:43.449846 28041 leader_election.cc:302] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a997ea4b0a7b49e8b95c8a2456c62c61, f4f41b8e5c3141279dd8ff3b1e7e74ec; no voters:
I20260812 19:03:43.450019 28235 raft_consensus.cc:2781] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:03:43.450085 28235 raft_consensus.cc:482] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:03:43.450130 28235 raft_consensus.cc:3037] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:43.451220 28235 raft_consensus.cc:504] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } }
I20260812 19:03:43.451577 28235 leader_election.cc:288] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec [CANDIDATE]: Term 1 election: Requested vote from peers a997ea4b0a7b49e8b95c8a2456c62c61 (127.25.130.1:42525), 38e8f436073b4f5494a52113d186a0c1 (127.25.130.2:46273)
I20260812 19:03:43.451980 27771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14463a5cfa924dbd8493979192a30284" candidate_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61"
I20260812 19:03:43.451997 27955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14463a5cfa924dbd8493979192a30284" candidate_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38e8f436073b4f5494a52113d186a0c1"
I20260812 19:03:43.452100 27771 raft_consensus.cc:3037] T 14463a5cfa924dbd8493979192a30284 P a997ea4b0a7b49e8b95c8a2456c62c61 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:43.452145 27955 raft_consensus.cc:3037] T 14463a5cfa924dbd8493979192a30284 P 38e8f436073b4f5494a52113d186a0c1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:43.453035 27771 raft_consensus.cc:2454] T 14463a5cfa924dbd8493979192a30284 P a997ea4b0a7b49e8b95c8a2456c62c61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4f41b8e5c3141279dd8ff3b1e7e74ec in term 1.
I20260812 19:03:43.453068 27955 raft_consensus.cc:2454] T 14463a5cfa924dbd8493979192a30284 P 38e8f436073b4f5494a52113d186a0c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4f41b8e5c3141279dd8ff3b1e7e74ec in term 1.
I20260812 19:03:43.453369 28041 leader_election.cc:302] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a997ea4b0a7b49e8b95c8a2456c62c61, f4f41b8e5c3141279dd8ff3b1e7e74ec; no voters:
I20260812 19:03:43.453570 28235 raft_consensus.cc:2781] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:43.453668 28235 raft_consensus.cc:686] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 1 LEADER]: Becoming Leader. State: Replica: f4f41b8e5c3141279dd8ff3b1e7e74ec, State: Running, Role: LEADER
I20260812 19:03:43.453811 28235 consensus_queue.cc:237] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } }
I20260812 19:03:43.454461 28220 tablet_replica.cc:439] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec: TxnStatusTablet state changed. Reason: New leader f4f41b8e5c3141279dd8ff3b1e7e74ec. Latest consensus state: current_term: 1 leader_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } } }
I20260812 19:03:43.454555 28220 tablet_replica.cc:442] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec: This TxnStatusTablet replica's current role is: LEADER
I20260812 19:03:43.454840 28262 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 19:03:43.455065 28262 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 19:03:43.455663 27560 catalog_manager.cc:5495] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec reported cstate change: term changed from 0 to 1, leader changed from <none> to f4f41b8e5c3141279dd8ff3b1e7e74ec (127.25.130.3). New cstate: current_term: 1 leader_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } health_report { overall_health: HEALTHY } } }
I20260812 19:03:43.473820 28256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:43.470232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:38817 (local address 127.0.0.1:39632)
0812 19:03:43.470380 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:43.470397 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:43.470494 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:43.470768 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:43.470770 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:43.470776 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 19:03:43.470874 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:43.470879 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:43.471858 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:43.471860 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:43.472492 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:43.472501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:43.473092 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:43.473096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:43.473174 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:43.473228 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 19:03:43.473549 (+ 321us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 19:03:43.473562 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 19:03:43.473608 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 19:03:43.473697 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 19:03:43.473831 28209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:43.470333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39632 (local address 127.25.130.3:38817)
0812 19:03:43.470470 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:43.470474 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:43.470491 (+ 17us) server_negotiation.cc:407] Connection header received
0812 19:03:43.470612 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:43.470615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:43.470672 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:43.470756 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:03:43.470957 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:43.471762 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:43.472608 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:43.472950 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:43.473078 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:43.473358 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:03:43.473469 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:03:43.473534 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:43.473622 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 19:03:43.473683 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 19:03:43.476953 27955 raft_consensus.cc:1261] T 14463a5cfa924dbd8493979192a30284 P 38e8f436073b4f5494a52113d186a0c1 [term 1 FOLLOWER]: Refusing update from remote peer f4f41b8e5c3141279dd8ff3b1e7e74ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:03:43.476953 27771 raft_consensus.cc:1261] T 14463a5cfa924dbd8493979192a30284 P a997ea4b0a7b49e8b95c8a2456c62c61 [term 1 FOLLOWER]: Refusing update from remote peer f4f41b8e5c3141279dd8ff3b1e7e74ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:03:43.477389 28220 consensus_queue.cc:1035] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:03:43.477561 28235 consensus_queue.cc:1035] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:03:43.478874 28217 tablet_replica.cc:439] T 14463a5cfa924dbd8493979192a30284 P a997ea4b0a7b49e8b95c8a2456c62c61: TxnStatusTablet state changed. Reason: New leader f4f41b8e5c3141279dd8ff3b1e7e74ec. Latest consensus state: current_term: 1 leader_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } } }
I20260812 19:03:43.478993 28217 tablet_replica.cc:442] T 14463a5cfa924dbd8493979192a30284 P a997ea4b0a7b49e8b95c8a2456c62c61: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 19:03:43.478945 28216 tablet_replica.cc:439] T 14463a5cfa924dbd8493979192a30284 P 38e8f436073b4f5494a52113d186a0c1: TxnStatusTablet state changed. Reason: New leader f4f41b8e5c3141279dd8ff3b1e7e74ec. Latest consensus state: current_term: 1 leader_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } } }
I20260812 19:03:43.479071 28216 tablet_replica.cc:442] T 14463a5cfa924dbd8493979192a30284 P 38e8f436073b4f5494a52113d186a0c1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 19:03:43.481019 28218 tablet_replica.cc:439] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } } }
I20260812 19:03:43.481149 28218 tablet_replica.cc:442] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec: This TxnStatusTablet replica's current role is: LEADER
I20260812 19:03:43.481382 28224 mvcc.cc:205] Tried to move back new op lower bound from 7317755590557147136 to 7317755590468202496. Current Snapshot: MvccSnapshot[applied={T|T < 7317755590557147136}]
I20260812 19:03:43.481509 28217 tablet_replica.cc:439] T 14463a5cfa924dbd8493979192a30284 P a997ea4b0a7b49e8b95c8a2456c62c61: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } } }
I20260812 19:03:43.481638 28217 tablet_replica.cc:442] T 14463a5cfa924dbd8493979192a30284 P a997ea4b0a7b49e8b95c8a2456c62c61: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 19:03:43.482117 28227 mvcc.cc:205] Tried to move back new op lower bound from 7317755590557147136 to 7317755590468202496. Current Snapshot: MvccSnapshot[applied={T|T < 7317755590557147136 or (T in {7317755590557147136})}]
I20260812 19:03:43.485107 28235 tablet_replica.cc:439] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } } }
I20260812 19:03:43.485260 28235 tablet_replica.cc:442] T 14463a5cfa924dbd8493979192a30284 P f4f41b8e5c3141279dd8ff3b1e7e74ec: This TxnStatusTablet replica's current role is: LEADER
I20260812 19:03:43.486449 28216 tablet_replica.cc:439] T 14463a5cfa924dbd8493979192a30284 P 38e8f436073b4f5494a52113d186a0c1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 } } peers { permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 } } peers { permanent_uuid: "f4f41b8e5c3141279dd8ff3b1e7e74ec" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38817 } } }
I20260812 19:03:43.486521 28228 mvcc.cc:205] Tried to move back new op lower bound from 7317755590557147136 to 7317755590468202496. Current Snapshot: MvccSnapshot[applied={T|T < 7317755590557147136}]
I20260812 19:03:43.486588 28216 tablet_replica.cc:442] T 14463a5cfa924dbd8493979192a30284 P 38e8f436073b4f5494a52113d186a0c1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 19:03:43.498857 28270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:43.495439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:38817 (local address 127.25.130.3:34101)
0812 19:03:43.495748 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:43.495759 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:43.495866 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:43.496069 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:43.496071 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:43.496079 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 19:03:43.496222 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:43.496227 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:43.496970 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:43.496974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:43.497711 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:43.497722 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:43.498128 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:43.498141 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:43.498220 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:43.498267 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 19:03:43.498531 (+ 264us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 19:03:43.498537 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 19:03:43.498600 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 19:03:43.498713 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":56,"thread_start_us":137,"threads_started":1}
I20260812 19:03:43.498857 28209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:43.495502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:34101 (local address 127.25.130.3:38817)
0812 19:03:43.495612 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:43.495617 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:43.495824 (+ 207us) server_negotiation.cc:407] Connection header received
0812 19:03:43.495963 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:43.495966 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:43.496001 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:43.496055 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:03:43.496309 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:43.496870 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:43.497822 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:43.498035 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:43.498116 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:43.498388 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:03:43.498462 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:03:43.498518 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:43.498608 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 19:03:43.498693 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":35}
I20260812 19:03:43.506592 27955 raft_consensus.cc:1261] T 91f4ae1567d84b48b36931bebf59650d P 38e8f436073b4f5494a52113d186a0c1 [term 1 FOLLOWER]: Refusing update from remote peer f4f41b8e5c3141279dd8ff3b1e7e74ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:03:43.506614 27771 raft_consensus.cc:1261] T 91f4ae1567d84b48b36931bebf59650d P a997ea4b0a7b49e8b95c8a2456c62c61 [term 1 FOLLOWER]: Refusing update from remote peer f4f41b8e5c3141279dd8ff3b1e7e74ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:03:43.507077 28220 consensus_queue.cc:1035] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38e8f436073b4f5494a52113d186a0c1" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 46273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:03:43.507265 28218 consensus_queue.cc:1035] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "a997ea4b0a7b49e8b95c8a2456c62c61" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 42525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:03:43.545820 27560 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:03:43.545995 27560 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:03:43.548118 27560 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7278dd442364a00b761d7cd3d3467ae: Sending DeleteTablet for 3 replicas of tablet 91f4ae1567d84b48b36931bebf59650d
I20260812 19:03:43.548839 27746 tablet_service.cc:1514] Processing DeleteTablet for tablet 91f4ae1567d84b48b36931bebf59650d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:03:43 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:42856
I20260812 19:03:43.549043 28243 tablet_replica.cc:330] T 91f4ae1567d84b48b36931bebf59650d P a997ea4b0a7b49e8b95c8a2456c62c61: stopping tablet replica
I20260812 19:03:43.549153 28243 raft_consensus.cc:2229] T 91f4ae1567d84b48b36931bebf59650d P a997ea4b0a7b49e8b95c8a2456c62c61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:03:43.549275 28243 raft_consensus.cc:2258] T 91f4ae1567d84b48b36931bebf59650d P a997ea4b0a7b49e8b95c8a2456c62c61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:03:43.549481 28102 tablet_service.cc:1514] Processing DeleteTablet for tablet 91f4ae1567d84b48b36931bebf59650d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:03:43 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:39610
I20260812 19:03:43.549674 28241 tablet_replica.cc:330] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec: stopping tablet replica
I20260812 19:03:43.549767 28241 raft_consensus.cc:2229] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:03:43.549937 28241 raft_consensus.cc:2258] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:03:43.550179 28243 ts_tablet_manager.cc:1905] T 91f4ae1567d84b48b36931bebf59650d P a997ea4b0a7b49e8b95c8a2456c62c61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:03:43.550283 27927 tablet_service.cc:1514] Processing DeleteTablet for tablet 91f4ae1567d84b48b36931bebf59650d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:03:43 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:49808
I20260812 19:03:43.550459 28242 tablet_replica.cc:330] T 91f4ae1567d84b48b36931bebf59650d P 38e8f436073b4f5494a52113d186a0c1: stopping tablet replica
I20260812 19:03:43.550570 28242 raft_consensus.cc:2229] T 91f4ae1567d84b48b36931bebf59650d P 38e8f436073b4f5494a52113d186a0c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:03:43.550686 28242 raft_consensus.cc:2258] T 91f4ae1567d84b48b36931bebf59650d P 38e8f436073b4f5494a52113d186a0c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:03:43.550889 28241 ts_tablet_manager.cc:1905] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:03:43.551402 28242 ts_tablet_manager.cc:1905] T 91f4ae1567d84b48b36931bebf59650d P 38e8f436073b4f5494a52113d186a0c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:03:43.552760 28243 ts_tablet_manager.cc:1918] T 91f4ae1567d84b48b36931bebf59650d P a997ea4b0a7b49e8b95c8a2456c62c61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 19:03:43.552824 28243 log.cc:1196] T 91f4ae1567d84b48b36931bebf59650d P a997ea4b0a7b49e8b95c8a2456c62c61: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-0/wal/wals/91f4ae1567d84b48b36931bebf59650d
I20260812 19:03:43.553007 28243 ts_tablet_manager.cc:1939] T 91f4ae1567d84b48b36931bebf59650d P a997ea4b0a7b49e8b95c8a2456c62c61: Deleting consensus metadata
I20260812 19:03:43.553742 27540 catalog_manager.cc:4841] TS a997ea4b0a7b49e8b95c8a2456c62c61 (127.25.130.1:42525): tablet 91f4ae1567d84b48b36931bebf59650d (table test-table [id=98fcfcc48bd94690a5e0eeaf8b54a778]) successfully deleted
I20260812 19:03:43.554136 28241 ts_tablet_manager.cc:1918] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 19:03:43.554227 28241 log.cc:1196] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-2/wal/wals/91f4ae1567d84b48b36931bebf59650d
I20260812 19:03:43.554246 28242 ts_tablet_manager.cc:1918] T 91f4ae1567d84b48b36931bebf59650d P 38e8f436073b4f5494a52113d186a0c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 19:03:43.554319 28242 log.cc:1196] T 91f4ae1567d84b48b36931bebf59650d P 38e8f436073b4f5494a52113d186a0c1: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561415988404-26120-0/minicluster-data/ts-1/wal/wals/91f4ae1567d84b48b36931bebf59650d
I20260812 19:03:43.554484 28241 ts_tablet_manager.cc:1939] T 91f4ae1567d84b48b36931bebf59650d P f4f41b8e5c3141279dd8ff3b1e7e74ec: Deleting consensus metadata
I20260812 19:03:43.554571 28242 ts_tablet_manager.cc:1939] T 91f4ae1567d84b48b36931bebf59650d P 38e8f436073b4f5494a52113d186a0c1: Deleting consensus metadata
I20260812 19:03:43.555258 27543 catalog_manager.cc:4841] TS 38e8f436073b4f5494a52113d186a0c1 (127.25.130.2:46273): tablet 91f4ae1567d84b48b36931bebf59650d (table test-table [id=98fcfcc48bd94690a5e0eeaf8b54a778]) successfully deleted
I20260812 19:03:43.555346 27540 catalog_manager.cc:4841] TS f4f41b8e5c3141279dd8ff3b1e7e74ec (127.25.130.3:38817): tablet 91f4ae1567d84b48b36931bebf59650d (table test-table [id=98fcfcc48bd94690a5e0eeaf8b54a778]) successfully deleted
Aug 12 19:03:43 dist-test-slave-49h3 krb5kdc[27506](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561422, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.130.1@KRBTEST.COM
I20260812 19:03:43.565541 28277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:43.553611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42872 (local address 127.25.130.1:42525)
0812 19:03:43.553928 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:43.553933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:43.555508 (+ 1575us) server_negotiation.cc:407] Connection header received
0812 19:03:43.555557 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:43.555560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:43.555622 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:43.555704 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:43.556442 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:43.557247 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:43.558120 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:43.558474 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:43.558591 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:43.561324 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:43.561346 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:43.561367 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:43.561403 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:43.563447 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:43.563942 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:43.563950 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:43.563952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:43.564067 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:43.564310 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:43.564314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:43.564315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:43.564702 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:43.564981 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:43.565158 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 19:03:43.565310 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":104,"threads_started":1}
W20260812 19:03:43.566248 27821 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42872
I20260812 19:03:43.576004 27627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:43.568146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60694 (local address 127.25.130.62:38489)
0812 19:03:43.568286 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:43.568290 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:43.568305 (+ 15us) server_negotiation.cc:407] Connection header received
0812 19:03:43.568420 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:43.568424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:43.568473 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:43.568557 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:43.569128 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:43.569950 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:43.570751 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:43.571091 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:43.571202 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:43.572497 (+ 1295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:43.572502 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:43.572504 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:43.572533 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:43.574147 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:43.574646 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:43.574650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:43.574652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:43.574756 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:43.575073 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:43.575076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:43.575078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:43.575335 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:43.575478 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:43.575698 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 19:03:43.575837 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260812 19:03:43.576459 27560 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60694
I20260812 19:03:43.577745 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 27657
I20260812 19:03:43.588768 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 27840
I20260812 19:03:43.611507 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 28022
I20260812 19:03:43.622068 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 27521
2026-08-12T19:03:43Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2890 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:03:43 dist-test-slave-49h3 krb5kdc[28284](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:03:43 dist-test-slave-49h3 krb5kdc[28284](info): set up 2 sockets
Aug 12 19:03:43 dist-test-slave-49h3 krb5kdc[28284](info): commencing operation
krb5kdc: starting...
W20260812 19:03:44.707008 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:03:44 dist-test-slave-49h3 krb5kdc[28284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:03:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:03:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:03:44.853222 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:38757
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:44683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:38757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:44.991607 28302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:44.992017 28302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:44.992087 28302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:44.996686 28302 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:44.996801 28302 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 19:03:44.996829 28302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:44.996855 28302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:44.996877 28302 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:44.996898 28302 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:45.003295 28302 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:44683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:38757
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:38757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:45.004895 28302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:45.006376 28302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:45.016970 28311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:45.016968 28312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:45.017067 28302 server_base.cc:1034] running on GCE node
W20260812 19:03:45.016968 28315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:45.018349 28302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:45.019765 28302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:45.020957 28302 hybrid_clock.cc:648] HybridClock initialized: now 1786561425020930 us; error 47 us; skew 500 ppm
Aug 12 19:03:45 dist-test-slave-49h3 krb5kdc[28284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561425, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:45.024118 28302 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:03:45.025238 28302 webserver.cc:460] Webserver started at http://127.25.130.62:39231/ using document root <none> and password file <none>
I20260812 19:03:45.025942 28302 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:45.026041 28302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:45.026295 28302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:45.028760 28302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "5996bc47f2674a18a22d0e59dee1cf32"
format_stamp: "Formatted at 2026-08-12 19:03:45 on dist-test-slave-49h3"
server_key: "9cb5499564a87bf86af755b46a28160f"
server_key_iv: "470937345ea7af9af22318b166d65bac"
server_key_version: "encryptionkey@0"
I20260812 19:03:45.029475 28302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "5996bc47f2674a18a22d0e59dee1cf32"
format_stamp: "Formatted at 2026-08-12 19:03:45 on dist-test-slave-49h3"
server_key: "9cb5499564a87bf86af755b46a28160f"
server_key_iv: "470937345ea7af9af22318b166d65bac"
server_key_version: "encryptionkey@0"
I20260812 19:03:45.033988 28302 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 19:03:45.037082 28323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:45.038616 28302 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:03:45.038774 28302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "5996bc47f2674a18a22d0e59dee1cf32"
format_stamp: "Formatted at 2026-08-12 19:03:45 on dist-test-slave-49h3"
server_key: "9cb5499564a87bf86af755b46a28160f"
server_key_iv: "470937345ea7af9af22318b166d65bac"
server_key_version: "encryptionkey@0"
I20260812 19:03:45.038908 28302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:45.054481 28302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:45.057926 28302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:45.058135 28302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:45.066524 28302 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:38757
I20260812 19:03:45.066531 28405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:38757 every 8 connection(s)
I20260812 19:03:45.067598 28302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:45.070710 28406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:45.073035 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 28302
I20260812 19:03:45.073180 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:03:45.073510 26120 external_mini_cluster.cc:1428] Setting key b69f63bf4e8251d240dd7f9e40023c25
I20260812 19:03:45.077608 28406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5996bc47f2674a18a22d0e59dee1cf32: Bootstrap starting.
I20260812 19:03:45.080117 28406 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5996bc47f2674a18a22d0e59dee1cf32: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:45.080917 28406 log.cc:824] T 00000000000000000000000000000000 P 5996bc47f2674a18a22d0e59dee1cf32: Log is configured to *not* fsync() on all Append() calls
Aug 12 19:03:45 dist-test-slave-49h3 krb5kdc[28284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:45.083446 28406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5996bc47f2674a18a22d0e59dee1cf32: No bootstrap required, opened a new log
I20260812 19:03:45.087832 28406 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5996bc47f2674a18a22d0e59dee1cf32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5996bc47f2674a18a22d0e59dee1cf32" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 38757 } }
I20260812 19:03:45.088099 28406 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5996bc47f2674a18a22d0e59dee1cf32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:45.088137 28406 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5996bc47f2674a18a22d0e59dee1cf32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5996bc47f2674a18a22d0e59dee1cf32, State: Initialized, Role: FOLLOWER
I20260812 19:03:45.088618 28406 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5996bc47f2674a18a22d0e59dee1cf32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5996bc47f2674a18a22d0e59dee1cf32" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 38757 } }
I20260812 19:03:45.088763 28406 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5996bc47f2674a18a22d0e59dee1cf32 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:03:45.088829 28406 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5996bc47f2674a18a22d0e59dee1cf32 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:03:45.088827 28409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:45.075179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49394 (local address 127.25.130.62:38757)
0812 19:03:45.075689 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:45.075706 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:45.075745 (+ 39us) server_negotiation.cc:407] Connection header received
0812 19:03:45.076553 (+ 808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:45.076574 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:45.076942 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:45.077283 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:45.078184 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.079193 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.080067 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.080627 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.080779 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:45.083683 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:45.083696 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:45.083718 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:45.083749 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:45.085547 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:45.086088 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:45.086093 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:45.086097 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:45.086228 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:45.086525 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:45.086528 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:45.086529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:45.086962 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:45.087149 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:45.087461 (+ 312us) server_negotiation.cc:299] Negotiation successful
0812 19:03:45.087801 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":167,"threads_started":1}
I20260812 19:03:45.088944 28406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5996bc47f2674a18a22d0e59dee1cf32 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:45.090040 28406 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5996bc47f2674a18a22d0e59dee1cf32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5996bc47f2674a18a22d0e59dee1cf32" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 38757 } }
I20260812 19:03:45.090445 28406 leader_election.cc:302] T 00000000000000000000000000000000 P 5996bc47f2674a18a22d0e59dee1cf32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5996bc47f2674a18a22d0e59dee1cf32; no voters:
I20260812 19:03:45.090850 28406 leader_election.cc:288] T 00000000000000000000000000000000 P 5996bc47f2674a18a22d0e59dee1cf32 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:03:45.090955 28411 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5996bc47f2674a18a22d0e59dee1cf32 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:45.091209 28411 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5996bc47f2674a18a22d0e59dee1cf32 [term 1 LEADER]: Becoming Leader. State: Replica: 5996bc47f2674a18a22d0e59dee1cf32, State: Running, Role: LEADER
I20260812 19:03:45.091594 28411 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5996bc47f2674a18a22d0e59dee1cf32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5996bc47f2674a18a22d0e59dee1cf32" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 38757 } }
I20260812 19:03:45.092183 28406 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5996bc47f2674a18a22d0e59dee1cf32 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:45.093328 28412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5996bc47f2674a18a22d0e59dee1cf32 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5996bc47f2674a18a22d0e59dee1cf32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5996bc47f2674a18a22d0e59dee1cf32" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 38757 } } }
I20260812 19:03:45.093500 28412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5996bc47f2674a18a22d0e59dee1cf32 [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:45.093909 28422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:45.094419 28413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5996bc47f2674a18a22d0e59dee1cf32 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5996bc47f2674a18a22d0e59dee1cf32. Latest consensus state: current_term: 1 leader_uuid: "5996bc47f2674a18a22d0e59dee1cf32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5996bc47f2674a18a22d0e59dee1cf32" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 38757 } } }
I20260812 19:03:45.094682 28413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5996bc47f2674a18a22d0e59dee1cf32 [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:45.097028 28422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:03:45.102895 28422 catalog_manager.cc:1347] Generated new cluster ID: d94a7200af2c49baa53d3a11d7ffd0ef
I20260812 19:03:45.102991 28422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:45.113545 28422 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:45.114609 28422 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:45.121240 28422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5996bc47f2674a18a22d0e59dee1cf32: Generated new TSK 0
I20260812 19:03:45.122081 28422 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
I20260812 19:03:45.160782 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:0
--local_ip_for_outbound_sockets=127.25.130.1
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:38757
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:44683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:45.284165 28436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:45.284485 28436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:45.284557 28436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:45.288750 28436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:45.288839 28436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:45.288895 28436 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 19:03:45.288924 28436 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 19:03:45.288981 28436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:03:45.294867 28436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:44683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.130.62:38757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:45.296110 28436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:45.297260 28436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:45.307929 28444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:45.308076 28436 server_base.cc:1034] running on GCE node
W20260812 19:03:45.307929 28443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:45.307929 28446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:45.308693 28436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:45.309408 28436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:45.310603 28436 hybrid_clock.cc:648] HybridClock initialized: now 1786561425310588 us; error 30 us; skew 500 ppm
Aug 12 19:03:45 dist-test-slave-49h3 krb5kdc[28284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561425, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:45.313592 28436 init.cc:377] Logged in from keytab as kudu/127.25.130.1@KRBTEST.COM (short username kudu)
I20260812 19:03:45.314654 28436 webserver.cc:460] Webserver started at http://127.25.130.1:42135/ using document root <none> and password file <none>
I20260812 19:03:45.315253 28436 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:45.315322 28436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:45.315544 28436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:45.317790 28436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-0/data/instance:
uuid: "e3a45ba920064206b98fbb462d8d4172"
format_stamp: "Formatted at 2026-08-12 19:03:45 on dist-test-slave-49h3"
server_key: "148b5212ebcdb74936ecb883a8dc0ed6"
server_key_iv: "aee3eb8438289922e1689ec5641d4a94"
server_key_version: "encryptionkey@0"
I20260812 19:03:45.318385 28436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance:
uuid: "e3a45ba920064206b98fbb462d8d4172"
format_stamp: "Formatted at 2026-08-12 19:03:45 on dist-test-slave-49h3"
server_key: "148b5212ebcdb74936ecb883a8dc0ed6"
server_key_iv: "aee3eb8438289922e1689ec5641d4a94"
server_key_version: "encryptionkey@0"
I20260812 19:03:45.322391 28436 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 19:03:45.324892 28454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:45.326275 28436 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:03:45.326395 28436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "e3a45ba920064206b98fbb462d8d4172"
format_stamp: "Formatted at 2026-08-12 19:03:45 on dist-test-slave-49h3"
server_key: "148b5212ebcdb74936ecb883a8dc0ed6"
server_key_iv: "aee3eb8438289922e1689ec5641d4a94"
server_key_version: "encryptionkey@0"
I20260812 19:03:45.326514 28436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:45.352766 28436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:45.356086 28436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:45.356290 28436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:45.357965 28436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:03:45.358022 28436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:45.358065 28436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:03:45.358083 28436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 19:03:45 dist-test-slave-49h3 krb5kdc[28284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561425, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:45.368953 28436 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:36715
I20260812 19:03:45.369079 28597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:36715 every 8 connection(s)
I20260812 19:03:45.369946 28436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:03:45.372215 28409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:45.358825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:60653 (local address 127.25.130.62:38757)
0812 19:03:45.358945 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:45.358949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:45.360193 (+ 1244us) server_negotiation.cc:407] Connection header received
0812 19:03:45.361300 (+ 1107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:45.361304 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:45.361375 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:45.361515 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:45.362902 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.363448 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.365208 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.365578 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.365684 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:45.368517 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:45.368521 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:45.368523 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:45.368587 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:45.370164 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:45.370788 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:45.370792 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:45.370794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:45.370898 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:45.371248 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:45.371250 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:45.371251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:45.371448 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:45.371519 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:45.371969 (+ 450us) server_negotiation.cc:299] Negotiation successful
0812 19:03:45.372085 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 19:03:45.373093 28465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:45.359443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:38757 (local address 127.25.130.1:60653)
0812 19:03:45.359989 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:45.360036 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:45.361041 (+ 1005us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:45.361721 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:45.361729 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:45.362124 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:45.362731 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:45.362748 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.363582 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.363585 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:45.365037 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:45.365051 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.365748 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.365753 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:45.365939 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:45.366567 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:45.366585 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:45.368301 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:45.370286 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:45.370293 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:45.370311 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:45.370642 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:45.371009 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:45.371012 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:45.371013 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:45.371169 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:45.371528 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:45.371533 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:45.371782 (+ 249us) client_negotiation.cc:769] Sending connection context
0812 19:03:45.371935 (+ 153us) client_negotiation.cc:240] Negotiation successful
0812 19:03:45.372242 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":158,"threads_started":1}
I20260812 19:03:45.374486 28598 heartbeater.cc:347] Connected to a master server at 127.25.130.62:38757
I20260812 19:03:45.374655 28598 heartbeater.cc:464] Registering TS with master...
I20260812 19:03:45.375269 28598 heartbeater.cc:511] Master 127.25.130.62:38757 requested a full tablet report, sending...
I20260812 19:03:45.377027 28344 ts_manager.cc:194] Registered new tserver with Master: e3a45ba920064206b98fbb462d8d4172 (127.25.130.1:36715)
I20260812 19:03:45.378499 28344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.1@KRBTEST.COM'} at 127.25.130.1:60653
I20260812 19:03:45.378980 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 28436
I20260812 19:03:45.379106 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance
I20260812 19:03:45.379446 26120 external_mini_cluster.cc:1428] Setting key 3ea17838c1e79d631cc692a982f624fc
I20260812 19:03:45.384145 28409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:45.377010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:47119 (local address 127.25.130.62:38757)
0812 19:03:45.377159 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:45.377163 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:45.377360 (+ 197us) server_negotiation.cc:407] Connection header received
0812 19:03:45.377512 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:45.377515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:45.377561 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:45.377629 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:45.378130 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.378666 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.379601 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.379910 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.380006 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:45.381281 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:45.381285 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:45.381288 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:45.381317 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:45.382580 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:45.382982 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:45.382986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:45.382989 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:45.383073 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:45.383332 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:45.383335 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:45.383337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:45.383525 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:45.383613 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:45.383870 (+ 257us) server_negotiation.cc:299] Negotiation successful
0812 19:03:45.383986 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 19:03:45.384157 28604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:45.376919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:38757 (local address 127.25.130.1:47119)
0812 19:03:45.377279 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:45.377294 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:45.377400 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:45.377662 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:45.377664 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:45.377895 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:45.378029 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:45.378037 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.378790 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.378793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:45.379465 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:45.379483 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.380025 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.380029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:45.380127 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:45.380684 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:45.380699 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:45.381138 (+ 439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:45.382685 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:45.382688 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:45.382690 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:45.382895 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:45.383154 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:45.383157 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:45.383160 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:45.383252 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:45.383613 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:45.383616 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:45.383715 (+ 99us) client_negotiation.cc:769] Sending connection context
0812 19:03:45.383801 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 19:03:45.383961 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":103,"threads_started":1}
WARNING: no policy specified for kudu/127.25.130.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
I20260812 19:03:45.409047 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:0
--local_ip_for_outbound_sockets=127.25.130.2
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:38757
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:44683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:45.524799 28607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:45.525104 28607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:45.525156 28607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:45.529335 28607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:45.529425 28607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:45.529507 28607 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 19:03:45.529551 28607 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 19:03:45.529601 28607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:03:45.535315 28607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:44683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.130.62:38757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:45.536558 28607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:45.537747 28607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:45.547807 28617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:45.547811 28614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:45.547807 28615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:45.547851 28607 server_base.cc:1034] running on GCE node
I20260812 19:03:45.548616 28607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:45.549309 28607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:45.550472 28607 hybrid_clock.cc:648] HybridClock initialized: now 1786561425550457 us; error 34 us; skew 500 ppm
Aug 12 19:03:45 dist-test-slave-49h3 krb5kdc[28284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561425, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:45.557641 28607 init.cc:377] Logged in from keytab as kudu/127.25.130.2@KRBTEST.COM (short username kudu)
I20260812 19:03:45.558708 28607 webserver.cc:460] Webserver started at http://127.25.130.2:42711/ using document root <none> and password file <none>
I20260812 19:03:45.559315 28607 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:45.559391 28607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:45.559633 28607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:45.561903 28607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-1/data/instance:
uuid: "271fd951ac344d53831f5f23e1decb78"
format_stamp: "Formatted at 2026-08-12 19:03:45 on dist-test-slave-49h3"
server_key: "d28303be908b5ae2181c8fda687f427a"
server_key_iv: "5bc19c875f7930f967dc9fbd1471060a"
server_key_version: "encryptionkey@0"
I20260812 19:03:45.562508 28607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance:
uuid: "271fd951ac344d53831f5f23e1decb78"
format_stamp: "Formatted at 2026-08-12 19:03:45 on dist-test-slave-49h3"
server_key: "d28303be908b5ae2181c8fda687f427a"
server_key_iv: "5bc19c875f7930f967dc9fbd1471060a"
server_key_version: "encryptionkey@0"
I20260812 19:03:45.566532 28607 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 19:03:45.569141 28625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:45.570487 28607 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:03:45.570614 28607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "271fd951ac344d53831f5f23e1decb78"
format_stamp: "Formatted at 2026-08-12 19:03:45 on dist-test-slave-49h3"
server_key: "d28303be908b5ae2181c8fda687f427a"
server_key_iv: "5bc19c875f7930f967dc9fbd1471060a"
server_key_version: "encryptionkey@0"
I20260812 19:03:45.570730 28607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:45.584321 28607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:45.587810 28607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:45.588006 28607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:45.589573 28607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:03:45.589633 28607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:45.589690 28607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:03:45.589716 28607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 19:03:45 dist-test-slave-49h3 krb5kdc[28284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561425, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:45.600075 28607 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:36689
I20260812 19:03:45.600107 28768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:36689 every 8 connection(s)
I20260812 19:03:45.601197 28607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:03:45.605193 28409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:45.590390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:49889 (local address 127.25.130.62:38757)
0812 19:03:45.590544 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:45.590549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:45.591465 (+ 916us) server_negotiation.cc:407] Connection header received
0812 19:03:45.592651 (+ 1186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:45.592655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:45.592722 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:45.592846 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:45.594460 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.595294 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.597073 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.597410 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.597568 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:45.600881 (+ 3313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:45.600885 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:45.600887 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:45.600920 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:45.602604 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:45.603153 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:45.603157 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:45.603159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:45.603258 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:45.603756 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:45.603760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:45.603762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:45.604049 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:45.604161 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:45.604843 (+ 682us) server_negotiation.cc:299] Negotiation successful
0812 19:03:45.605004 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 19:03:45.606371 28635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:45.590815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:38757 (local address 127.25.130.2:49889)
0812 19:03:45.591291 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:45.591342 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:45.592383 (+ 1041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:45.593035 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:45.593045 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:45.593553 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:45.594284 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:45.594306 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.595451 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.595455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:45.596935 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:45.596947 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.597621 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.597626 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:45.597819 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:45.598740 (+ 921us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:45.598763 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:45.600650 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:45.602733 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:45.602738 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:45.602750 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:45.603024 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:45.603406 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:45.603410 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:45.603412 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:45.603623 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:45.604178 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:45.604186 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:45.604488 (+ 302us) client_negotiation.cc:769] Sending connection context
0812 19:03:45.604716 (+ 228us) client_negotiation.cc:240] Negotiation successful
0812 19:03:45.605098 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":114,"threads_started":1}
I20260812 19:03:45.606967 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 28607
I20260812 19:03:45.607077 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance
I20260812 19:03:45.607381 26120 external_mini_cluster.cc:1428] Setting key f8a92994baa170c83236a5f042556850
I20260812 19:03:45.608917 28769 heartbeater.cc:347] Connected to a master server at 127.25.130.62:38757
I20260812 19:03:45.609104 28769 heartbeater.cc:464] Registering TS with master...
I20260812 19:03:45.609829 28769 heartbeater.cc:511] Master 127.25.130.62:38757 requested a full tablet report, sending...
I20260812 19:03:45.613663 28345 ts_manager.cc:194] Registered new tserver with Master: 271fd951ac344d53831f5f23e1decb78 (127.25.130.2:36689)
I20260812 19:03:45.614399 28345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.2@KRBTEST.COM'} at 127.25.130.2:49889
WARNING: no policy specified for kudu/127.25.130.3@KRBTEST.COM; defaulting to no policy
I20260812 19:03:45.628764 28778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:45.617086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:38757 (local address 127.25.130.2:53999)
0812 19:03:45.617380 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:45.617394 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:45.617545 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:45.617800 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:45.617803 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:45.618055 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:45.618199 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:45.618207 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.620227 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.620231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:45.621128 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:45.621155 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.623585 (+ 2430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.623589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:45.623702 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:45.624343 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:45.624358 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:45.624723 (+ 365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:45.626711 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:45.626715 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:45.626717 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:45.627004 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:45.627389 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:45.627393 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:45.627395 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:45.627511 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:45.628170 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:45.628173 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:45.628275 (+ 102us) client_negotiation.cc:769] Sending connection context
0812 19:03:45.628366 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 19:03:45.628553 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":97,"threads_started":1}
I20260812 19:03:45.628758 28409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:45.616933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:53999 (local address 127.25.130.62:38757)
0812 19:03:45.617111 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:45.617117 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:45.617500 (+ 383us) server_negotiation.cc:407] Connection header received
0812 19:03:45.617648 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:45.617652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:45.617700 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:45.617767 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:45.618379 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.619169 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.621295 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.623406 (+ 2111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.623533 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:45.624888 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:45.624891 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:45.624893 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:45.624924 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:45.626529 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:45.627127 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:45.627131 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:45.627133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:45.627251 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:45.627634 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:45.627637 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:45.627640 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:45.628003 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:45.628121 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:45.628457 (+ 336us) server_negotiation.cc:299] Negotiation successful
0812 19:03:45.628591 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
I20260812 19:03:45.649340 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:0
--local_ip_for_outbound_sockets=127.25.130.3
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:38757
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:44683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:45.768560 28781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:45.768874 28781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:45.768926 28781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:45.773098 28781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:45.773190 28781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:45.773234 28781 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 19:03:45.773248 28781 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 19:03:45.773288 28781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:03:45.779161 28781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:44683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.130.62:38757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:45.780333 28781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:45.781394 28781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:45.791121 28787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:45.791121 28789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:45.791126 28791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:45.791563 28781 server_base.cc:1034] running on GCE node
I20260812 19:03:45.792007 28781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:45.792765 28781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:45.793927 28781 hybrid_clock.cc:648] HybridClock initialized: now 1786561425793894 us; error 47 us; skew 500 ppm
Aug 12 19:03:45 dist-test-slave-49h3 krb5kdc[28284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561425, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:45.796711 28781 init.cc:377] Logged in from keytab as kudu/127.25.130.3@KRBTEST.COM (short username kudu)
I20260812 19:03:45.797679 28781 webserver.cc:460] Webserver started at http://127.25.130.3:33769/ using document root <none> and password file <none>
I20260812 19:03:45.798257 28781 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:45.798328 28781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:45.798554 28781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:45.800681 28781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-2/data/instance:
uuid: "dcdceefb4b834db0958d23434f10492f"
format_stamp: "Formatted at 2026-08-12 19:03:45 on dist-test-slave-49h3"
server_key: "0c35470d11d849c234167ff5ea1f803f"
server_key_iv: "97b095da1b62b7330873b1c9dd3e8b88"
server_key_version: "encryptionkey@0"
I20260812 19:03:45.801265 28781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance:
uuid: "dcdceefb4b834db0958d23434f10492f"
format_stamp: "Formatted at 2026-08-12 19:03:45 on dist-test-slave-49h3"
server_key: "0c35470d11d849c234167ff5ea1f803f"
server_key_iv: "97b095da1b62b7330873b1c9dd3e8b88"
server_key_version: "encryptionkey@0"
I20260812 19:03:45.805208 28781 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260812 19:03:45.807938 28798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:45.809370 28781 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 19:03:45.809531 28781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "dcdceefb4b834db0958d23434f10492f"
format_stamp: "Formatted at 2026-08-12 19:03:45 on dist-test-slave-49h3"
server_key: "0c35470d11d849c234167ff5ea1f803f"
server_key_iv: "97b095da1b62b7330873b1c9dd3e8b88"
server_key_version: "encryptionkey@0"
I20260812 19:03:45.809667 28781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:45.827127 28781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:45.830255 28781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:45.830456 28781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:45.832053 28781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:03:45.832110 28781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:45.832155 28781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:03:45.832185 28781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 19:03:45 dist-test-slave-49h3 krb5kdc[28284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561425, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:45.842996 28781 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:42037
I20260812 19:03:45.843521 28944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:42037 every 8 connection(s)
I20260812 19:03:45.844072 28781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:03:45.846312 28409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:45.832965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:46185 (local address 127.25.130.62:38757)
0812 19:03:45.833090 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:45.833093 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:45.834323 (+ 1230us) server_negotiation.cc:407] Connection header received
0812 19:03:45.835485 (+ 1162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:45.835489 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:45.835556 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:45.835648 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:45.837385 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.838226 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.839512 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.839748 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.839843 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:45.842522 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:45.842526 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:45.842528 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:45.842558 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:45.844198 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:45.844823 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:45.844827 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:45.844829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:45.844932 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:45.845263 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:45.845266 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:45.845267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:45.845508 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:45.845594 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:45.846065 (+ 471us) server_negotiation.cc:299] Negotiation successful
0812 19:03:45.846185 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 19:03:45.847244 28810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:45.833608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:38757 (local address 127.25.130.3:46185)
0812 19:03:45.834157 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:45.834205 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:45.835250 (+ 1045us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:45.835887 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:45.835897 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:45.836410 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:45.837157 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:45.837178 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.838353 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.838357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:45.839380 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:45.839392 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.839902 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.839906 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:45.840075 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:45.840660 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:45.840678 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:45.842331 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:45.844296 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:45.844302 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:45.844318 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:45.844690 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:45.845025 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:45.845028 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:45.845029 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:45.845177 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:45.845611 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:45.845616 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:45.845863 (+ 247us) client_negotiation.cc:769] Sending connection context
0812 19:03:45.846034 (+ 171us) client_negotiation.cc:240] Negotiation successful
0812 19:03:45.846309 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":154,"threads_started":1}
I20260812 19:03:45.847790 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 28781
I20260812 19:03:45.847904 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance
I20260812 19:03:45.848230 26120 external_mini_cluster.cc:1428] Setting key 261f6d273bf263e81e3c55dfc035aa15
I20260812 19:03:45.848465 28945 heartbeater.cc:347] Connected to a master server at 127.25.130.62:38757
I20260812 19:03:45.848619 28945 heartbeater.cc:464] Registering TS with master...
I20260812 19:03:45.849196 28945 heartbeater.cc:511] Master 127.25.130.62:38757 requested a full tablet report, sending...
I20260812 19:03:45.850391 28344 ts_manager.cc:194] Registered new tserver with Master: dcdceefb4b834db0958d23434f10492f (127.25.130.3:42037)
I20260812 19:03:45.851135 28344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.3@KRBTEST.COM'} at 127.25.130.3:46185
I20260812 19:03:45.857221 28954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:45.850856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:38757 (local address 127.25.130.3:51443)
0812 19:03:45.851132 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:45.851144 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:45.851238 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:45.851460 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:45.851463 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:45.851688 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:45.851813 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:45.851819 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.852824 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.852827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:45.853411 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:45.853420 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.853852 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.853855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:45.853927 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:45.854407 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:45.854418 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:45.854757 (+ 339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:45.856055 (+ 1298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:45.856057 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:45.856059 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:45.856250 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:45.856443 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:45.856444 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:45.856446 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:45.856510 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:45.856784 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:45.856786 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:45.856861 (+ 75us) client_negotiation.cc:769] Sending connection context
0812 19:03:45.856933 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 19:03:45.857047 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":88,"threads_started":1}
I20260812 19:03:45.857282 28409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:45.850907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:51443 (local address 127.25.130.62:38757)
0812 19:03:45.851022 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:45.851026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:45.851209 (+ 183us) server_negotiation.cc:407] Connection header received
0812 19:03:45.851311 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:45.851314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:45.851360 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:45.851431 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:45.851929 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.852687 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.853551 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.853767 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.853843 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:45.854861 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:45.854863 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:45.854865 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:45.854881 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:45.855968 (+ 1087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:45.856318 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:45.856321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:45.856322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:45.856386 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:45.856565 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:45.856567 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:45.856568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:45.856723 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:45.856792 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:45.857056 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 19:03:45.857163 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 19:03:45.862278 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 19:03:45 dist-test-slave-49h3 krb5kdc[28284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 19:03:45 dist-test-slave-49h3 krb5kdc[28284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:45.889848 28409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:45.881358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59052 (local address 127.25.130.62:38757)
0812 19:03:45.881570 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:45.881573 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:45.881587 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:03:45.881634 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:45.881637 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:45.881678 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:45.881758 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:45.882546 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.883110 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.883975 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.884209 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.884302 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:45.886916 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:45.886919 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:45.886921 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:45.886950 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:45.888247 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:45.888661 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:45.888663 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:45.888664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:45.888740 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:45.889053 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:45.889055 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:45.889057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:45.889240 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:45.889334 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:45.889592 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 19:03:45.889715 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 19:03:45.892920 28344 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59052:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:03:45.896153 28344 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:03:45.912271 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:45.905705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:42037 (local address 127.0.0.1:57314)
0812 19:03:45.906529 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:45.906588 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:45.906752 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:45.907639 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:45.907645 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:45.907668 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:03:45.907832 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:45.907841 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.909006 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.909009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:45.910387 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:45.910398 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.911590 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.911594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:45.911716 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:45.911800 (+ 84us) client_negotiation.cc:769] Sending connection context
0812 19:03:45.911911 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 19:03:45.912079 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"mutex_wait_us":56,"thread_start_us":105,"threads_started":1}
I20260812 19:03:45.912292 28971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:45.906625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41476 (local address 127.25.130.1:36715)
0812 19:03:45.907406 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:45.907413 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:45.907437 (+ 24us) server_negotiation.cc:407] Connection header received
0812 19:03:45.907610 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:45.907616 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:45.907802 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:45.907942 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:03:45.908345 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.909538 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.910964 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.911509 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.911611 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:45.911774 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:45.912045 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 19:03:45.912171 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":76,"threads_started":1}
I20260812 19:03:45.912335 28972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:45.906641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44676 (local address 127.25.130.2:36689)
0812 19:03:45.907068 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:45.907081 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:45.907140 (+ 59us) server_negotiation.cc:407] Connection header received
0812 19:03:45.907552 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:45.907558 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:45.907749 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:45.907905 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:03:45.908124 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.909316 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.910930 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.911778 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.911890 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:45.912055 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:45.912109 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 19:03:45.912204 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":116,"threads_started":1}
I20260812 19:03:45.912688 28967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:45.906142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:36715 (local address 127.0.0.1:41476)
0812 19:03:45.907356 (+ 1214us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:45.907369 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:45.907445 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:45.908091 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:45.908094 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:45.908113 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:03:45.908214 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:45.908220 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.909675 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.909678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:45.910842 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:45.910850 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.911637 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.911641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:45.911747 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:45.911804 (+ 57us) client_negotiation.cc:769] Sending connection context
0812 19:03:45.911920 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 19:03:45.912548 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1157,"mutex_wait_us":499,"thread_start_us":98,"threads_started":1}
I20260812 19:03:45.912278 28970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:45.905880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:36689 (local address 127.0.0.1:44676)
0812 19:03:45.907054 (+ 1174us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:45.907067 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:45.907165 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:45.907883 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:45.907886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:45.907905 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:03:45.908000 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:45.908006 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.909496 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.909500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:45.910781 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:45.910793 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.911886 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.911889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:45.911976 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:45.911997 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 19:03:45.912051 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 19:03:45.912121 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1096,"mutex_wait_us":23,"thread_start_us":115,"threads_started":1}
I20260812 19:03:45.913148 28968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:45.905854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57314 (local address 127.25.130.3:42037)
0812 19:03:45.907069 (+ 1215us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:45.907081 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:45.907107 (+ 26us) server_negotiation.cc:407] Connection header received
0812 19:03:45.907188 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:45.907192 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:45.907428 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:45.907681 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:03:45.907971 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.908880 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.910526 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.911210 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.911318 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:45.911505 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:45.912901 (+ 1396us) server_negotiation.cc:299] Negotiation successful
0812 19:03:45.913024 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1104,"thread_start_us":143,"threads_started":1}
I20260812 19:03:45.916077 28687 tablet_service.cc:1467] Processing CreateTablet for tablet 8ab5b3b32cfb4f19b344c16b175a8c76 (DEFAULT_TABLE table=test-table [id=64060264db774f008200e848ad7e4423]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:03:45.916081 28861 tablet_service.cc:1467] Processing CreateTablet for tablet 8ab5b3b32cfb4f19b344c16b175a8c76 (DEFAULT_TABLE table=test-table [id=64060264db774f008200e848ad7e4423]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:03:45.916080 28517 tablet_service.cc:1467] Processing CreateTablet for tablet 8ab5b3b32cfb4f19b344c16b175a8c76 (DEFAULT_TABLE table=test-table [id=64060264db774f008200e848ad7e4423]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:03:45.917112 28687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ab5b3b32cfb4f19b344c16b175a8c76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:45.917142 28517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ab5b3b32cfb4f19b344c16b175a8c76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:45.917141 28861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ab5b3b32cfb4f19b344c16b175a8c76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:45.924739 28973 tablet_bootstrap.cc:492] T 8ab5b3b32cfb4f19b344c16b175a8c76 P 271fd951ac344d53831f5f23e1decb78: Bootstrap starting.
I20260812 19:03:45.926168 28977 tablet_bootstrap.cc:492] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f: Bootstrap starting.
I20260812 19:03:45.927186 28973 tablet_bootstrap.cc:654] T 8ab5b3b32cfb4f19b344c16b175a8c76 P 271fd951ac344d53831f5f23e1decb78: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:45.927951 28973 log.cc:824] T 8ab5b3b32cfb4f19b344c16b175a8c76 P 271fd951ac344d53831f5f23e1decb78: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:45.928006 28977 tablet_bootstrap.cc:654] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:45.928809 28974 tablet_bootstrap.cc:492] T 8ab5b3b32cfb4f19b344c16b175a8c76 P e3a45ba920064206b98fbb462d8d4172: Bootstrap starting.
I20260812 19:03:45.928823 28977 log.cc:824] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:45.930827 28973 tablet_bootstrap.cc:492] T 8ab5b3b32cfb4f19b344c16b175a8c76 P 271fd951ac344d53831f5f23e1decb78: No bootstrap required, opened a new log
I20260812 19:03:45.930927 28977 tablet_bootstrap.cc:492] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f: No bootstrap required, opened a new log
I20260812 19:03:45.931046 28973 ts_tablet_manager.cc:1397] T 8ab5b3b32cfb4f19b344c16b175a8c76 P 271fd951ac344d53831f5f23e1decb78: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 19:03:45.931128 28977 ts_tablet_manager.cc:1397] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 19:03:45.931190 28974 tablet_bootstrap.cc:654] T 8ab5b3b32cfb4f19b344c16b175a8c76 P e3a45ba920064206b98fbb462d8d4172: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:45.932055 28974 log.cc:824] T 8ab5b3b32cfb4f19b344c16b175a8c76 P e3a45ba920064206b98fbb462d8d4172: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:45.934319 28974 tablet_bootstrap.cc:492] T 8ab5b3b32cfb4f19b344c16b175a8c76 P e3a45ba920064206b98fbb462d8d4172: No bootstrap required, opened a new log
I20260812 19:03:45.934538 28974 ts_tablet_manager.cc:1397] T 8ab5b3b32cfb4f19b344c16b175a8c76 P e3a45ba920064206b98fbb462d8d4172: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 19:03:45.934422 28973 raft_consensus.cc:348] T 8ab5b3b32cfb4f19b344c16b175a8c76 P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } }
I20260812 19:03:45.934692 28973 raft_consensus.cc:374] T 8ab5b3b32cfb4f19b344c16b175a8c76 P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:45.934744 28973 raft_consensus.cc:729] T 8ab5b3b32cfb4f19b344c16b175a8c76 P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 271fd951ac344d53831f5f23e1decb78, State: Initialized, Role: FOLLOWER
I20260812 19:03:45.935199 28973 consensus_queue.cc:260] T 8ab5b3b32cfb4f19b344c16b175a8c76 P 271fd951ac344d53831f5f23e1decb78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } }
I20260812 19:03:45.936287 28973 ts_tablet_manager.cc:1428] T 8ab5b3b32cfb4f19b344c16b175a8c76 P 271fd951ac344d53831f5f23e1decb78: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 19:03:45.936672 28769 heartbeater.cc:503] Master 127.25.130.62:38757 was elected leader, sending a full tablet report...
I20260812 19:03:45.939181 28977 raft_consensus.cc:348] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } }
I20260812 19:03:45.939400 28974 raft_consensus.cc:348] T 8ab5b3b32cfb4f19b344c16b175a8c76 P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } }
I20260812 19:03:45.939620 28974 raft_consensus.cc:374] T 8ab5b3b32cfb4f19b344c16b175a8c76 P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:45.939620 28977 raft_consensus.cc:374] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:45.939678 28974 raft_consensus.cc:729] T 8ab5b3b32cfb4f19b344c16b175a8c76 P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3a45ba920064206b98fbb462d8d4172, State: Initialized, Role: FOLLOWER
I20260812 19:03:45.939678 28977 raft_consensus.cc:729] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcdceefb4b834db0958d23434f10492f, State: Initialized, Role: FOLLOWER
I20260812 19:03:45.940164 28974 consensus_queue.cc:260] T 8ab5b3b32cfb4f19b344c16b175a8c76 P e3a45ba920064206b98fbb462d8d4172 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } }
I20260812 19:03:45.940164 28977 consensus_queue.cc:260] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } }
I20260812 19:03:45.941263 28974 ts_tablet_manager.cc:1428] T 8ab5b3b32cfb4f19b344c16b175a8c76 P e3a45ba920064206b98fbb462d8d4172: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 19:03:45.941263 28977 ts_tablet_manager.cc:1428] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20260812 19:03:45.941668 28598 heartbeater.cc:503] Master 127.25.130.62:38757 was elected leader, sending a full tablet report...
I20260812 19:03:45.942157 28945 heartbeater.cc:503] Master 127.25.130.62:38757 was elected leader, sending a full tablet report...
I20260812 19:03:45.955265 28985 raft_consensus.cc:482] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:03:45.955538 28985 raft_consensus.cc:504] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } }
I20260812 19:03:45.956888 28985 leader_election.cc:288] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 271fd951ac344d53831f5f23e1decb78 (127.25.130.2:36689), e3a45ba920064206b98fbb462d8d4172 (127.25.130.1:36715)
I20260812 19:03:45.960630 28810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:45.957139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:36689 (local address 127.25.130.3:34601)
0812 19:03:45.957295 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:45.957311 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:45.957408 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:45.957700 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:45.957703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:45.957723 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:03:45.957855 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:45.957863 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.958667 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.958670 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:45.959617 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:45.959628 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.960261 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.960265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:45.960368 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:45.960389 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 19:03:45.960442 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 19:03:45.960505 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 19:03:45.960705 28972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:45.957277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:34601 (local address 127.25.130.2:36689)
0812 19:03:45.957425 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:45.957430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:45.957471 (+ 41us) server_negotiation.cc:407] Connection header received
0812 19:03:45.957531 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:45.957534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:45.957585 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:45.957675 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:03:45.957962 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.958555 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.959737 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.960148 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.960246 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:45.960306 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:45.960470 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 19:03:45.960568 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 19:03:45.961381 28971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:45.957351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:49083 (local address 127.25.130.1:36715)
0812 19:03:45.957537 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:45.957541 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:45.957880 (+ 339us) server_negotiation.cc:407] Connection header received
0812 19:03:45.957973 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:45.957976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:45.958023 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:45.958092 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:03:45.958690 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.959180 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.960306 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.960869 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.960982 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:45.961030 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:45.961221 (+ 191us) server_negotiation.cc:299] Negotiation successful
0812 19:03:45.961285 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 19:03:45.961411 28986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:45.957141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:36715 (local address 127.25.130.3:49083)
0812 19:03:45.957784 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:45.957798 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:45.957893 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:45.958104 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:45.958106 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:45.958119 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:03:45.958570 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:45.958578 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.959295 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.959298 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:45.960176 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:45.960185 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:45.960991 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:45.960994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:45.961095 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:45.961121 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 19:03:45.961192 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 19:03:45.961271 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"mutex_wait_us":261,"thread_start_us":141,"threads_started":1}
I20260812 19:03:45.961409 28714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ab5b3b32cfb4f19b344c16b175a8c76" candidate_uuid: "dcdceefb4b834db0958d23434f10492f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "271fd951ac344d53831f5f23e1decb78" is_pre_election: true
I20260812 19:03:45.961797 28714 raft_consensus.cc:2454] T 8ab5b3b32cfb4f19b344c16b175a8c76 P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcdceefb4b834db0958d23434f10492f in term 0.
I20260812 19:03:45.962136 28545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ab5b3b32cfb4f19b344c16b175a8c76" candidate_uuid: "dcdceefb4b834db0958d23434f10492f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3a45ba920064206b98fbb462d8d4172" is_pre_election: true
I20260812 19:03:45.962325 28800 leader_election.cc:302] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 271fd951ac344d53831f5f23e1decb78, dcdceefb4b834db0958d23434f10492f; no voters:
I20260812 19:03:45.962421 28545 raft_consensus.cc:2454] T 8ab5b3b32cfb4f19b344c16b175a8c76 P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcdceefb4b834db0958d23434f10492f in term 0.
I20260812 19:03:45.962606 28985 raft_consensus.cc:2781] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:03:45.962697 28985 raft_consensus.cc:482] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:03:45.962744 28985 raft_consensus.cc:3037] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:45.963888 28985 raft_consensus.cc:504] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } }
I20260812 19:03:45.964257 28985 leader_election.cc:288] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f [CANDIDATE]: Term 1 election: Requested vote from peers 271fd951ac344d53831f5f23e1decb78 (127.25.130.2:36689), e3a45ba920064206b98fbb462d8d4172 (127.25.130.1:36715)
I20260812 19:03:45.964695 28545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ab5b3b32cfb4f19b344c16b175a8c76" candidate_uuid: "dcdceefb4b834db0958d23434f10492f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3a45ba920064206b98fbb462d8d4172"
I20260812 19:03:45.964679 28714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ab5b3b32cfb4f19b344c16b175a8c76" candidate_uuid: "dcdceefb4b834db0958d23434f10492f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "271fd951ac344d53831f5f23e1decb78"
I20260812 19:03:45.964828 28714 raft_consensus.cc:3037] T 8ab5b3b32cfb4f19b344c16b175a8c76 P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:45.964828 28545 raft_consensus.cc:3037] T 8ab5b3b32cfb4f19b344c16b175a8c76 P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:45.966271 28545 raft_consensus.cc:2454] T 8ab5b3b32cfb4f19b344c16b175a8c76 P e3a45ba920064206b98fbb462d8d4172 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcdceefb4b834db0958d23434f10492f in term 1.
I20260812 19:03:45.966588 28714 raft_consensus.cc:2454] T 8ab5b3b32cfb4f19b344c16b175a8c76 P 271fd951ac344d53831f5f23e1decb78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcdceefb4b834db0958d23434f10492f in term 1.
I20260812 19:03:45.966670 28804 leader_election.cc:302] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dcdceefb4b834db0958d23434f10492f, e3a45ba920064206b98fbb462d8d4172; no voters:
I20260812 19:03:45.966879 28985 raft_consensus.cc:2781] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:45.967116 28985 raft_consensus.cc:686] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f [term 1 LEADER]: Becoming Leader. State: Replica: dcdceefb4b834db0958d23434f10492f, State: Running, Role: LEADER
I20260812 19:03:45.967486 28985 consensus_queue.cc:237] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } }
I20260812 19:03:45.970903 28343 catalog_manager.cc:5495] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f reported cstate change: term changed from 0 to 1, leader changed from <none> to dcdceefb4b834db0958d23434f10492f (127.25.130.3). New cstate: current_term: 1 leader_uuid: "dcdceefb4b834db0958d23434f10492f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } health_report { overall_health: UNKNOWN } } }
Aug 12 19:03:46 dist-test-slave-49h3 krb5kdc[28284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561425, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:46.031899 28994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:46.023454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:38757 (local address 127.0.0.1:59062)
0812 19:03:46.023811 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:46.023824 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:46.023946 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:46.024209 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:46.024213 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:46.024661 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:46.024771 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:46.024778 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.025559 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.025562 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:46.026132 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:46.026141 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.026559 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.026562 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:46.026642 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:46.027186 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:46.027197 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:46.028741 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:46.030310 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:46.030323 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:46.030330 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:46.030575 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:46.030818 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:46.030821 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:46.030823 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:46.030904 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:46.031274 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:46.031283 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:46.031403 (+ 120us) client_negotiation.cc:769] Sending connection context
0812 19:03:46.031506 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 19:03:46.031691 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":163,"threads_started":1}
I20260812 19:03:46.031898 28409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:46.023529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59062 (local address 127.25.130.62:38757)
0812 19:03:46.023677 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:46.023682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:46.023900 (+ 218us) server_negotiation.cc:407] Connection header received
0812 19:03:46.024056 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:46.024059 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:46.024098 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:46.024167 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:46.024889 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.025416 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.026245 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.026451 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.026538 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:46.028883 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:46.028886 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:46.028888 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:46.028911 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:46.030206 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:46.030663 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:46.030667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:46.030669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:46.030741 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:46.030977 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:46.030980 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:46.030982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:46.031176 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:46.031263 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:46.031564 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 19:03:46.031772 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":90,"server-negotiator.queue_time_us":54}
I20260812 19:03:46.036033 28343 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:59062:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 19:03:46.036788 28343 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:03:46.041692 28687 tablet_service.cc:1467] Processing CreateTablet for tablet 6e8e8cff5645446e8378c89dd524a99a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=87753630e4af4ec8a6692071b90d3c48]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 19:03:46.041714 28861 tablet_service.cc:1467] Processing CreateTablet for tablet 6e8e8cff5645446e8378c89dd524a99a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=87753630e4af4ec8a6692071b90d3c48]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 19:03:46.041646 28517 tablet_service.cc:1467] Processing CreateTablet for tablet 6e8e8cff5645446e8378c89dd524a99a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=87753630e4af4ec8a6692071b90d3c48]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 19:03:46.042479 28517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e8e8cff5645446e8378c89dd524a99a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:46.042479 28861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e8e8cff5645446e8378c89dd524a99a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:46.042479 28687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e8e8cff5645446e8378c89dd524a99a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:46.045502 28974 tablet_bootstrap.cc:492] T 6e8e8cff5645446e8378c89dd524a99a P e3a45ba920064206b98fbb462d8d4172: Bootstrap starting.
I20260812 19:03:46.045761 28973 tablet_bootstrap.cc:492] T 6e8e8cff5645446e8378c89dd524a99a P 271fd951ac344d53831f5f23e1decb78: Bootstrap starting.
I20260812 19:03:46.046682 28974 tablet_bootstrap.cc:654] T 6e8e8cff5645446e8378c89dd524a99a P e3a45ba920064206b98fbb462d8d4172: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:46.046687 28973 tablet_bootstrap.cc:654] T 6e8e8cff5645446e8378c89dd524a99a P 271fd951ac344d53831f5f23e1decb78: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:46.047425 28977 tablet_bootstrap.cc:492] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f: Bootstrap starting.
I20260812 19:03:46.048137 28973 tablet_bootstrap.cc:492] T 6e8e8cff5645446e8378c89dd524a99a P 271fd951ac344d53831f5f23e1decb78: No bootstrap required, opened a new log
I20260812 19:03:46.048238 28973 ts_tablet_manager.cc:1397] T 6e8e8cff5645446e8378c89dd524a99a P 271fd951ac344d53831f5f23e1decb78: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 19:03:46.048383 28974 tablet_bootstrap.cc:492] T 6e8e8cff5645446e8378c89dd524a99a P e3a45ba920064206b98fbb462d8d4172: No bootstrap required, opened a new log
I20260812 19:03:46.048460 28977 tablet_bootstrap.cc:654] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:46.048471 28974 ts_tablet_manager.cc:1397] T 6e8e8cff5645446e8378c89dd524a99a P e3a45ba920064206b98fbb462d8d4172: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 19:03:46.048795 28973 raft_consensus.cc:348] T 6e8e8cff5645446e8378c89dd524a99a P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:46.048928 28973 raft_consensus.cc:374] T 6e8e8cff5645446e8378c89dd524a99a P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:46.048969 28973 raft_consensus.cc:729] T 6e8e8cff5645446e8378c89dd524a99a P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 271fd951ac344d53831f5f23e1decb78, State: Initialized, Role: FOLLOWER
I20260812 19:03:46.048943 28974 raft_consensus.cc:348] T 6e8e8cff5645446e8378c89dd524a99a P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:46.049082 28974 raft_consensus.cc:374] T 6e8e8cff5645446e8378c89dd524a99a P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:46.049124 28974 raft_consensus.cc:729] T 6e8e8cff5645446e8378c89dd524a99a P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3a45ba920064206b98fbb462d8d4172, State: Initialized, Role: FOLLOWER
I20260812 19:03:46.049151 28973 consensus_queue.cc:260] T 6e8e8cff5645446e8378c89dd524a99a P 271fd951ac344d53831f5f23e1decb78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:46.049248 28974 consensus_queue.cc:260] T 6e8e8cff5645446e8378c89dd524a99a P e3a45ba920064206b98fbb462d8d4172 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:46.049470 28982 tablet_replica.cc:439] T 6e8e8cff5645446e8378c89dd524a99a P 271fd951ac344d53831f5f23e1decb78: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } } }
I20260812 19:03:46.049642 28982 tablet_replica.cc:442] T 6e8e8cff5645446e8378c89dd524a99a P 271fd951ac344d53831f5f23e1decb78: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 19:03:46.049554 28984 tablet_replica.cc:439] T 6e8e8cff5645446e8378c89dd524a99a P e3a45ba920064206b98fbb462d8d4172: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } } }
I20260812 19:03:46.049715 28984 tablet_replica.cc:442] T 6e8e8cff5645446e8378c89dd524a99a P e3a45ba920064206b98fbb462d8d4172: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 19:03:46.049438 28973 ts_tablet_manager.cc:1428] T 6e8e8cff5645446e8378c89dd524a99a P 271fd951ac344d53831f5f23e1decb78: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 19:03:46.049540 28974 ts_tablet_manager.cc:1428] T 6e8e8cff5645446e8378c89dd524a99a P e3a45ba920064206b98fbb462d8d4172: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 19:03:46.051324 28977 tablet_bootstrap.cc:492] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f: No bootstrap required, opened a new log
I20260812 19:03:46.051419 28977 ts_tablet_manager.cc:1397] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 19:03:46.051891 28977 raft_consensus.cc:348] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:46.052014 28977 raft_consensus.cc:374] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:46.052064 28977 raft_consensus.cc:729] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcdceefb4b834db0958d23434f10492f, State: Initialized, Role: FOLLOWER
I20260812 19:03:46.052192 28977 consensus_queue.cc:260] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:46.052462 28977 ts_tablet_manager.cc:1428] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 19:03:46.052445 28985 tablet_replica.cc:439] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } } }
I20260812 19:03:46.052597 28985 tablet_replica.cc:442] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 19:03:46.083706 28982 raft_consensus.cc:482] T 6e8e8cff5645446e8378c89dd524a99a P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:03:46.083843 28982 raft_consensus.cc:504] T 6e8e8cff5645446e8378c89dd524a99a P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:46.084899 28982 leader_election.cc:288] T 6e8e8cff5645446e8378c89dd524a99a P 271fd951ac344d53831f5f23e1decb78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3a45ba920064206b98fbb462d8d4172 (127.25.130.1:36715), dcdceefb4b834db0958d23434f10492f (127.25.130.3:42037)
I20260812 19:03:46.086251 28985 raft_consensus.cc:482] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:03:46.086355 28985 raft_consensus.cc:504] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:46.086719 28985 leader_election.cc:288] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3a45ba920064206b98fbb462d8d4172 (127.25.130.1:36715), 271fd951ac344d53831f5f23e1decb78 (127.25.130.2:36689)
I20260812 19:03:46.087075 28545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e8e8cff5645446e8378c89dd524a99a" candidate_uuid: "dcdceefb4b834db0958d23434f10492f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3a45ba920064206b98fbb462d8d4172" is_pre_election: true
I20260812 19:03:46.087208 28545 raft_consensus.cc:2454] T 6e8e8cff5645446e8378c89dd524a99a P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcdceefb4b834db0958d23434f10492f in term 0.
I20260812 19:03:46.087276 28714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e8e8cff5645446e8378c89dd524a99a" candidate_uuid: "dcdceefb4b834db0958d23434f10492f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "271fd951ac344d53831f5f23e1decb78" is_pre_election: true
I20260812 19:03:46.087410 28714 raft_consensus.cc:2454] T 6e8e8cff5645446e8378c89dd524a99a P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcdceefb4b834db0958d23434f10492f in term 0.
I20260812 19:03:46.087509 28804 leader_election.cc:302] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dcdceefb4b834db0958d23434f10492f, e3a45ba920064206b98fbb462d8d4172; no voters:
I20260812 19:03:46.087828 28985 raft_consensus.cc:2781] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:03:46.087888 28985 raft_consensus.cc:482] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:03:46.087921 28985 raft_consensus.cc:3037] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:46.089154 28985 raft_consensus.cc:504] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:46.089540 28985 leader_election.cc:288] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f [CANDIDATE]: Term 1 election: Requested vote from peers e3a45ba920064206b98fbb462d8d4172 (127.25.130.1:36715), 271fd951ac344d53831f5f23e1decb78 (127.25.130.2:36689)
I20260812 19:03:46.089699 28968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:46.085227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:52407 (local address 127.25.130.3:42037)
0812 19:03:46.085440 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:46.085470 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:46.085742 (+ 272us) server_negotiation.cc:407] Connection header received
0812 19:03:46.085849 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:46.085852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:46.085897 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:46.085955 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:03:46.086226 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.086949 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.088452 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.089013 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.089147 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:46.089205 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:46.089497 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 19:03:46.089574 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260812 19:03:46.089726 28998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:46.085136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:42037 (local address 127.25.130.2:52407)
0812 19:03:46.085642 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:46.085654 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:46.085752 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:46.085982 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:46.085984 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:46.086000 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:03:46.086121 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:46.086127 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.087055 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.087057 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:46.088327 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:46.088338 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.089189 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.089192 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:46.089283 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:46.089312 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 19:03:46.089427 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 19:03:46.089562 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"mutex_wait_us":45,"thread_start_us":92,"threads_started":1}
I20260812 19:03:46.090022 28545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e8e8cff5645446e8378c89dd524a99a" candidate_uuid: "dcdceefb4b834db0958d23434f10492f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3a45ba920064206b98fbb462d8d4172"
I20260812 19:03:46.090157 28545 raft_consensus.cc:3037] T 6e8e8cff5645446e8378c89dd524a99a P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:46.090317 28888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e8e8cff5645446e8378c89dd524a99a" candidate_uuid: "271fd951ac344d53831f5f23e1decb78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcdceefb4b834db0958d23434f10492f" is_pre_election: true
I20260812 19:03:46.090524 28635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:46.085105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:36715 (local address 127.25.130.2:56211)
0812 19:03:46.085259 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:46.085275 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:46.085366 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:46.085774 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:46.085777 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:46.085792 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:03:46.085912 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:46.085917 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.086950 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.086954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:46.088328 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:46.088338 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.090119 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.090123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:46.090235 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:46.090259 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 19:03:46.090328 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 19:03:46.090406 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":128}
I20260812 19:03:46.090577 28888 raft_consensus.cc:2379] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 271fd951ac344d53831f5f23e1decb78 in current term 1: Already voted for candidate dcdceefb4b834db0958d23434f10492f in this term.
I20260812 19:03:46.090564 28714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e8e8cff5645446e8378c89dd524a99a" candidate_uuid: "dcdceefb4b834db0958d23434f10492f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "271fd951ac344d53831f5f23e1decb78"
I20260812 19:03:46.090675 28714 raft_consensus.cc:3037] T 6e8e8cff5645446e8378c89dd524a99a P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:46.090847 28971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:46.085440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:56211 (local address 127.25.130.1:36715)
0812 19:03:46.085579 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:46.085583 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:46.085594 (+ 11us) server_negotiation.cc:407] Connection header received
0812 19:03:46.085640 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:46.085642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:46.085678 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:46.085749 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:03:46.086035 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.086815 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.089271 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.089891 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.089993 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:46.090042 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:46.090659 (+ 617us) server_negotiation.cc:299] Negotiation successful
0812 19:03:46.090741 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 19:03:46.091307 28545 raft_consensus.cc:2454] T 6e8e8cff5645446e8378c89dd524a99a P e3a45ba920064206b98fbb462d8d4172 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcdceefb4b834db0958d23434f10492f in term 1.
I20260812 19:03:46.091609 28714 raft_consensus.cc:2454] T 6e8e8cff5645446e8378c89dd524a99a P 271fd951ac344d53831f5f23e1decb78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcdceefb4b834db0958d23434f10492f in term 1.
I20260812 19:03:46.091617 28804 leader_election.cc:302] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dcdceefb4b834db0958d23434f10492f, e3a45ba920064206b98fbb462d8d4172; no voters:
I20260812 19:03:46.091776 28985 raft_consensus.cc:2781] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:46.091849 28985 raft_consensus.cc:686] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f [term 1 LEADER]: Becoming Leader. State: Replica: dcdceefb4b834db0958d23434f10492f, State: Running, Role: LEADER
I20260812 19:03:46.091979 28985 consensus_queue.cc:237] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:46.092671 28545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e8e8cff5645446e8378c89dd524a99a" candidate_uuid: "271fd951ac344d53831f5f23e1decb78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3a45ba920064206b98fbb462d8d4172" is_pre_election: true
I20260812 19:03:46.092880 28545 raft_consensus.cc:2379] T 6e8e8cff5645446e8378c89dd524a99a P e3a45ba920064206b98fbb462d8d4172 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 271fd951ac344d53831f5f23e1decb78 in current term 1: Already voted for candidate dcdceefb4b834db0958d23434f10492f in this term.
I20260812 19:03:46.093401 28630 leader_election.cc:302] T 6e8e8cff5645446e8378c89dd524a99a P 271fd951ac344d53831f5f23e1decb78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 271fd951ac344d53831f5f23e1decb78; no voters: dcdceefb4b834db0958d23434f10492f, e3a45ba920064206b98fbb462d8d4172
I20260812 19:03:46.093663 28987 tablet_replica.cc:439] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f: TxnStatusTablet state changed. Reason: New leader dcdceefb4b834db0958d23434f10492f. Latest consensus state: current_term: 1 leader_uuid: "dcdceefb4b834db0958d23434f10492f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } } }
I20260812 19:03:46.093736 28982 raft_consensus.cc:2726] T 6e8e8cff5645446e8378c89dd524a99a P 271fd951ac344d53831f5f23e1decb78 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:03:46.093792 28987 tablet_replica.cc:442] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f: This TxnStatusTablet replica's current role is: LEADER
I20260812 19:03:46.094245 29001 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 19:03:46.094465 29001 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260812 19:03:46.095044 28947 tablet.cc:2369] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:03:46.095196 28345 catalog_manager.cc:5495] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f reported cstate change: term changed from 0 to 1, leader changed from <none> to dcdceefb4b834db0958d23434f10492f (127.25.130.3). New cstate: current_term: 1 leader_uuid: "dcdceefb4b834db0958d23434f10492f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } health_report { overall_health: HEALTHY } } }
W20260812 19:03:46.102772 28770 tablet.cc:2369] T 6e8e8cff5645446e8378c89dd524a99a P 271fd951ac344d53831f5f23e1decb78: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:03:46.117246 28968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:46.113311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57324 (local address 127.25.130.3:42037)
0812 19:03:46.113508 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:46.113513 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:46.113530 (+ 17us) server_negotiation.cc:407] Connection header received
0812 19:03:46.113672 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:46.113675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:46.113741 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:46.113839 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:03:46.114044 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.114840 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.115725 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.115944 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.116036 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:46.116355 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:03:46.116810 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:03:46.116916 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:46.117034 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 19:03:46.117094 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 19:03:46.117245 28994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:46.113191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:42037 (local address 127.0.0.1:57324)
0812 19:03:46.113357 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:46.113373 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:46.113523 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:46.113847 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:46.113849 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:46.113855 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 19:03:46.113956 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:46.113961 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.114973 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.114977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:46.115597 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:46.115608 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.116054 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.116058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:46.116158 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:46.116210 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 19:03:46.116911 (+ 701us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 19:03:46.116940 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 19:03:46.117004 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 19:03:46.117109 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 19:03:46.121611 28599 tablet.cc:2369] T 6e8e8cff5645446e8378c89dd524a99a P e3a45ba920064206b98fbb462d8d4172: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:03:46.122488 28714 raft_consensus.cc:1261] T 6e8e8cff5645446e8378c89dd524a99a P 271fd951ac344d53831f5f23e1decb78 [term 1 FOLLOWER]: Refusing update from remote peer dcdceefb4b834db0958d23434f10492f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:03:46.122937 28545 raft_consensus.cc:1261] T 6e8e8cff5645446e8378c89dd524a99a P e3a45ba920064206b98fbb462d8d4172 [term 1 FOLLOWER]: Refusing update from remote peer dcdceefb4b834db0958d23434f10492f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:03:46.123168 28987 consensus_queue.cc:1035] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f [LEADER]: Connected to new peer: Peer: permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:03:46.123454 28985 consensus_queue.cc:1035] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:03:46.127661 28982 tablet_replica.cc:439] T 6e8e8cff5645446e8378c89dd524a99a P 271fd951ac344d53831f5f23e1decb78: TxnStatusTablet state changed. Reason: New leader dcdceefb4b834db0958d23434f10492f. Latest consensus state: current_term: 1 leader_uuid: "dcdceefb4b834db0958d23434f10492f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } } }
I20260812 19:03:46.127803 28982 tablet_replica.cc:442] T 6e8e8cff5645446e8378c89dd524a99a P 271fd951ac344d53831f5f23e1decb78: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 19:03:46.127990 28984 tablet_replica.cc:439] T 6e8e8cff5645446e8378c89dd524a99a P e3a45ba920064206b98fbb462d8d4172: TxnStatusTablet state changed. Reason: New leader dcdceefb4b834db0958d23434f10492f. Latest consensus state: current_term: 1 leader_uuid: "dcdceefb4b834db0958d23434f10492f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } } }
I20260812 19:03:46.128120 28984 tablet_replica.cc:442] T 6e8e8cff5645446e8378c89dd524a99a P e3a45ba920064206b98fbb462d8d4172: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 19:03:46.130853 28985 tablet_replica.cc:439] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dcdceefb4b834db0958d23434f10492f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } } }
I20260812 19:03:46.130996 28985 tablet_replica.cc:442] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f: This TxnStatusTablet replica's current role is: LEADER
I20260812 19:03:46.131153 29005 mvcc.cc:205] Tried to move back new op lower bound from 7317755601390583808 to 7317755601274249216. Current Snapshot: MvccSnapshot[applied={T|T < 7317755601390583808}]
I20260812 19:03:46.131222 29008 mvcc.cc:205] Tried to move back new op lower bound from 7317755601390583808 to 7317755601274249216. Current Snapshot: MvccSnapshot[applied={T|T < 7317755601390583808}]
I20260812 19:03:46.131335 28982 tablet_replica.cc:439] T 6e8e8cff5645446e8378c89dd524a99a P 271fd951ac344d53831f5f23e1decb78: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dcdceefb4b834db0958d23434f10492f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } } }
I20260812 19:03:46.131438 28982 tablet_replica.cc:442] T 6e8e8cff5645446e8378c89dd524a99a P 271fd951ac344d53831f5f23e1decb78: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 19:03:46.138336 28987 tablet_replica.cc:439] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dcdceefb4b834db0958d23434f10492f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } } }
I20260812 19:03:46.138486 28987 tablet_replica.cc:442] T 6e8e8cff5645446e8378c89dd524a99a P dcdceefb4b834db0958d23434f10492f: This TxnStatusTablet replica's current role is: LEADER
I20260812 19:03:46.139806 29006 mvcc.cc:205] Tried to move back new op lower bound from 7317755601390583808 to 7317755601274249216. Current Snapshot: MvccSnapshot[applied={T|T < 7317755601390583808}]
I20260812 19:03:46.140550 28984 tablet_replica.cc:439] T 6e8e8cff5645446e8378c89dd524a99a P e3a45ba920064206b98fbb462d8d4172: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dcdceefb4b834db0958d23434f10492f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } } }
I20260812 19:03:46.140686 28984 tablet_replica.cc:442] T 6e8e8cff5645446e8378c89dd524a99a P e3a45ba920064206b98fbb462d8d4172: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 19:03:46.144989 28968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:46.141080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57338 (local address 127.25.130.3:42037)
0812 19:03:46.141274 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:46.141279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:46.141292 (+ 13us) server_negotiation.cc:407] Connection header received
0812 19:03:46.141329 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:46.141331 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:46.141371 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:46.141437 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:03:46.141784 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.142499 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.143865 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.144115 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.144202 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:46.144539 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:03:46.144650 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:03:46.144726 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:46.144829 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 19:03:46.144881 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260812 19:03:46.156477 28968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:46.153417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:44761 (local address 127.25.130.3:42037)
0812 19:03:46.153537 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:46.153541 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:46.153631 (+ 90us) server_negotiation.cc:407] Connection header received
0812 19:03:46.153761 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:46.153764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:46.153817 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:46.153876 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:03:46.154149 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.154640 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.155451 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.155658 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.155742 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:46.156034 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:03:46.156122 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:03:46.156179 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:46.156261 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 19:03:46.156312 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":66}
I20260812 19:03:46.156478 28954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:46.153360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:42037 (local address 127.25.130.3:44761)
0812 19:03:46.153552 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:46.153567 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:46.153677 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:46.153890 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:46.153893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:46.153899 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 19:03:46.154044 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:46.154049 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.154737 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.154740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:46.155344 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:46.155351 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.155752 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.155755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:46.155840 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:46.155887 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 19:03:46.156191 (+ 304us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 19:03:46.156196 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 19:03:46.156255 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 19:03:46.156340 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":6}
I20260812 19:03:46.163803 28714 raft_consensus.cc:1261] T 8ab5b3b32cfb4f19b344c16b175a8c76 P 271fd951ac344d53831f5f23e1decb78 [term 1 FOLLOWER]: Refusing update from remote peer dcdceefb4b834db0958d23434f10492f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:03:46.163803 28545 raft_consensus.cc:1261] T 8ab5b3b32cfb4f19b344c16b175a8c76 P e3a45ba920064206b98fbb462d8d4172 [term 1 FOLLOWER]: Refusing update from remote peer dcdceefb4b834db0958d23434f10492f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:03:46.164206 29014 consensus_queue.cc:1035] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f [LEADER]: Connected to new peer: Peer: permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:03:46.164372 28985 consensus_queue.cc:1035] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:03:46.209028 28345 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:03:46.209268 28345 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:03:46.212172 28345 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5996bc47f2674a18a22d0e59dee1cf32: Sending DeleteTablet for 3 replicas of tablet 8ab5b3b32cfb4f19b344c16b175a8c76
I20260812 19:03:46.213079 28861 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ab5b3b32cfb4f19b344c16b175a8c76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:03:46 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:57314
I20260812 19:03:46.213158 28517 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ab5b3b32cfb4f19b344c16b175a8c76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:03:46 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:41476
I20260812 19:03:46.213207 28687 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ab5b3b32cfb4f19b344c16b175a8c76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:03:46 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:44676
I20260812 19:03:46.213775 29029 tablet_replica.cc:330] T 8ab5b3b32cfb4f19b344c16b175a8c76 P e3a45ba920064206b98fbb462d8d4172: stopping tablet replica
I20260812 19:03:46.214018 29030 tablet_replica.cc:330] T 8ab5b3b32cfb4f19b344c16b175a8c76 P 271fd951ac344d53831f5f23e1decb78: stopping tablet replica
I20260812 19:03:46.214186 29029 raft_consensus.cc:2229] T 8ab5b3b32cfb4f19b344c16b175a8c76 P e3a45ba920064206b98fbb462d8d4172 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:03:46.214342 29030 raft_consensus.cc:2229] T 8ab5b3b32cfb4f19b344c16b175a8c76 P 271fd951ac344d53831f5f23e1decb78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:03:46.214461 29029 raft_consensus.cc:2258] T 8ab5b3b32cfb4f19b344c16b175a8c76 P e3a45ba920064206b98fbb462d8d4172 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:03:46.214550 29030 raft_consensus.cc:2258] T 8ab5b3b32cfb4f19b344c16b175a8c76 P 271fd951ac344d53831f5f23e1decb78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:03:46.214872 29028 tablet_replica.cc:330] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f: stopping tablet replica
I20260812 19:03:46.215193 29028 raft_consensus.cc:2229] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:03:46.215595 29028 raft_consensus.cc:2258] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:03:46.215471 28345 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59052:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:03:46.215999 28345 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:03:46.216439 29030 ts_tablet_manager.cc:1905] T 8ab5b3b32cfb4f19b344c16b175a8c76 P 271fd951ac344d53831f5f23e1decb78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:03:46.217655 29029 ts_tablet_manager.cc:1905] T 8ab5b3b32cfb4f19b344c16b175a8c76 P e3a45ba920064206b98fbb462d8d4172: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:03:46.219852 29030 ts_tablet_manager.cc:1918] T 8ab5b3b32cfb4f19b344c16b175a8c76 P 271fd951ac344d53831f5f23e1decb78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 19:03:46.219974 29030 log.cc:1196] T 8ab5b3b32cfb4f19b344c16b175a8c76 P 271fd951ac344d53831f5f23e1decb78: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-1/wal/wals/8ab5b3b32cfb4f19b344c16b175a8c76
I20260812 19:03:46.220333 29030 ts_tablet_manager.cc:1939] T 8ab5b3b32cfb4f19b344c16b175a8c76 P 271fd951ac344d53831f5f23e1decb78: Deleting consensus metadata
I20260812 19:03:46.221710 28517 tablet_service.cc:1467] Processing CreateTablet for tablet f90440cabc2c4d1ebe666503af8ffc10 (DEFAULT_TABLE table=test-table [id=62692c548c3a4b6abcc99c7179fb915e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:03:46.221836 28687 tablet_service.cc:1467] Processing CreateTablet for tablet f90440cabc2c4d1ebe666503af8ffc10 (DEFAULT_TABLE table=test-table [id=62692c548c3a4b6abcc99c7179fb915e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:03:46.222038 28517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f90440cabc2c4d1ebe666503af8ffc10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:46.222102 28687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f90440cabc2c4d1ebe666503af8ffc10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:46.224261 28974 tablet_bootstrap.cc:492] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172: Bootstrap starting.
I20260812 19:03:46.224337 28326 catalog_manager.cc:4841] TS 271fd951ac344d53831f5f23e1decb78 (127.25.130.2:36689): tablet 8ab5b3b32cfb4f19b344c16b175a8c76 (table test-table [id=64060264db774f008200e848ad7e4423]) successfully deleted
I20260812 19:03:46.224543 28973 tablet_bootstrap.cc:492] T f90440cabc2c4d1ebe666503af8ffc10 P 271fd951ac344d53831f5f23e1decb78: Bootstrap starting.
I20260812 19:03:46.225284 28974 tablet_bootstrap.cc:654] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:46.225281 28861 tablet_service.cc:1467] Processing CreateTablet for tablet f90440cabc2c4d1ebe666503af8ffc10 (DEFAULT_TABLE table=test-table [id=62692c548c3a4b6abcc99c7179fb915e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:03:46.225535 28973 tablet_bootstrap.cc:654] T f90440cabc2c4d1ebe666503af8ffc10 P 271fd951ac344d53831f5f23e1decb78: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:46.225584 28861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f90440cabc2c4d1ebe666503af8ffc10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:46.227233 28974 tablet_bootstrap.cc:492] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172: No bootstrap required, opened a new log
I20260812 19:03:46.227334 28974 ts_tablet_manager.cc:1397] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 19:03:46.227854 28974 raft_consensus.cc:348] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:46.227981 28974 raft_consensus.cc:374] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:46.228014 28974 raft_consensus.cc:729] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3a45ba920064206b98fbb462d8d4172, State: Initialized, Role: FOLLOWER
I20260812 19:03:46.228143 28974 consensus_queue.cc:260] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:46.228457 28974 ts_tablet_manager.cc:1428] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 19:03:46.228803 28977 tablet_bootstrap.cc:492] T f90440cabc2c4d1ebe666503af8ffc10 P dcdceefb4b834db0958d23434f10492f: Bootstrap starting.
I20260812 19:03:46.229825 28977 tablet_bootstrap.cc:654] T f90440cabc2c4d1ebe666503af8ffc10 P dcdceefb4b834db0958d23434f10492f: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:46.230445 28973 tablet_bootstrap.cc:492] T f90440cabc2c4d1ebe666503af8ffc10 P 271fd951ac344d53831f5f23e1decb78: No bootstrap required, opened a new log
I20260812 19:03:46.230551 28973 ts_tablet_manager.cc:1397] T f90440cabc2c4d1ebe666503af8ffc10 P 271fd951ac344d53831f5f23e1decb78: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260812 19:03:46.231051 28973 raft_consensus.cc:348] T f90440cabc2c4d1ebe666503af8ffc10 P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:46.231191 28973 raft_consensus.cc:374] T f90440cabc2c4d1ebe666503af8ffc10 P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:46.231225 28973 raft_consensus.cc:729] T f90440cabc2c4d1ebe666503af8ffc10 P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 271fd951ac344d53831f5f23e1decb78, State: Initialized, Role: FOLLOWER
I20260812 19:03:46.231390 29028 ts_tablet_manager.cc:1905] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:03:46.231364 28973 consensus_queue.cc:260] T f90440cabc2c4d1ebe666503af8ffc10 P 271fd951ac344d53831f5f23e1decb78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:46.231649 28973 ts_tablet_manager.cc:1428] T f90440cabc2c4d1ebe666503af8ffc10 P 271fd951ac344d53831f5f23e1decb78: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 19:03:46.233198 28977 tablet_bootstrap.cc:492] T f90440cabc2c4d1ebe666503af8ffc10 P dcdceefb4b834db0958d23434f10492f: No bootstrap required, opened a new log
I20260812 19:03:46.233289 28977 ts_tablet_manager.cc:1397] T f90440cabc2c4d1ebe666503af8ffc10 P dcdceefb4b834db0958d23434f10492f: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 19:03:46.233767 28977 raft_consensus.cc:348] T f90440cabc2c4d1ebe666503af8ffc10 P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:46.233875 28977 raft_consensus.cc:374] T f90440cabc2c4d1ebe666503af8ffc10 P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:46.233903 28977 raft_consensus.cc:729] T f90440cabc2c4d1ebe666503af8ffc10 P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcdceefb4b834db0958d23434f10492f, State: Initialized, Role: FOLLOWER
I20260812 19:03:46.234019 28977 consensus_queue.cc:260] T f90440cabc2c4d1ebe666503af8ffc10 P dcdceefb4b834db0958d23434f10492f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:46.234275 28977 ts_tablet_manager.cc:1428] T f90440cabc2c4d1ebe666503af8ffc10 P dcdceefb4b834db0958d23434f10492f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 19:03:46.234457 29028 ts_tablet_manager.cc:1918] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 19:03:46.234553 29028 log.cc:1196] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-2/wal/wals/8ab5b3b32cfb4f19b344c16b175a8c76
I20260812 19:03:46.234889 29028 ts_tablet_manager.cc:1939] T 8ab5b3b32cfb4f19b344c16b175a8c76 P dcdceefb4b834db0958d23434f10492f: Deleting consensus metadata
I20260812 19:03:46.235944 28327 catalog_manager.cc:4841] TS dcdceefb4b834db0958d23434f10492f (127.25.130.3:42037): tablet 8ab5b3b32cfb4f19b344c16b175a8c76 (table test-table [id=64060264db774f008200e848ad7e4423]) successfully deleted
I20260812 19:03:46.236704 29029 ts_tablet_manager.cc:1918] T 8ab5b3b32cfb4f19b344c16b175a8c76 P e3a45ba920064206b98fbb462d8d4172: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 19:03:46.236805 29029 log.cc:1196] T 8ab5b3b32cfb4f19b344c16b175a8c76 P e3a45ba920064206b98fbb462d8d4172: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-0/wal/wals/8ab5b3b32cfb4f19b344c16b175a8c76
I20260812 19:03:46.237156 29029 ts_tablet_manager.cc:1939] T 8ab5b3b32cfb4f19b344c16b175a8c76 P e3a45ba920064206b98fbb462d8d4172: Deleting consensus metadata
I20260812 19:03:46.238379 28328 catalog_manager.cc:4841] TS e3a45ba920064206b98fbb462d8d4172 (127.25.130.1:36715): tablet 8ab5b3b32cfb4f19b344c16b175a8c76 (table test-table [id=64060264db774f008200e848ad7e4423]) successfully deleted
I20260812 19:03:46.427179 28984 raft_consensus.cc:482] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:03:46.427366 28984 raft_consensus.cc:504] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:46.428507 28984 leader_election.cc:288] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 271fd951ac344d53831f5f23e1decb78 (127.25.130.2:36689), dcdceefb4b834db0958d23434f10492f (127.25.130.3:42037)
I20260812 19:03:46.434043 28972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:46.429119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:36249 (local address 127.25.130.2:36689)
0812 19:03:46.429257 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:46.429262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:46.429479 (+ 217us) server_negotiation.cc:407] Connection header received
0812 19:03:46.429595 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:46.429598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:46.429646 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:46.429716 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:03:46.429942 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.430745 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.432680 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.433323 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.433497 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:46.433560 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:46.433809 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 19:03:46.433892 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 19:03:46.434072 28968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:46.428883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:38427 (local address 127.25.130.3:42037)
0812 19:03:46.429010 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:46.429015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:46.429206 (+ 191us) server_negotiation.cc:407] Connection header received
0812 19:03:46.429342 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:46.429346 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:46.429407 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:46.429494 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:03:46.429886 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.430715 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.432680 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.433323 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.433499 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:46.433558 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:46.433809 (+ 251us) server_negotiation.cc:299] Negotiation successful
0812 19:03:46.433892 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 19:03:46.434165 29035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:46.428783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:36689 (local address 127.25.130.1:36249)
0812 19:03:46.429361 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:46.429375 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:46.429508 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:46.429744 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:46.429747 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:46.429760 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:03:46.429835 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:46.429840 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.430861 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.430865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:46.432509 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:46.432526 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.433495 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.433500 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:46.433648 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:46.433694 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 19:03:46.433771 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 19:03:46.433929 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"mutex_wait_us":229,"thread_start_us":83,"threads_started":1}
I20260812 19:03:46.434384 29034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:46.428783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:42037 (local address 127.25.130.1:38427)
0812 19:03:46.429099 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:46.429113 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:46.429238 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:46.429630 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:46.429640 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:46.429655 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:03:46.429773 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:46.429780 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.430834 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.430839 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:46.432509 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:46.432526 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.433495 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.433500 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:46.433652 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:46.433698 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 19:03:46.433770 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 19:03:46.433929 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":184,"thread_start_us":93,"threads_started":1}
I20260812 19:03:46.434875 28713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f90440cabc2c4d1ebe666503af8ffc10" candidate_uuid: "e3a45ba920064206b98fbb462d8d4172" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "271fd951ac344d53831f5f23e1decb78" is_pre_election: true
I20260812 19:03:46.435034 28713 raft_consensus.cc:2454] T f90440cabc2c4d1ebe666503af8ffc10 P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3a45ba920064206b98fbb462d8d4172 in term 0.
I20260812 19:03:46.435288 28888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f90440cabc2c4d1ebe666503af8ffc10" candidate_uuid: "e3a45ba920064206b98fbb462d8d4172" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcdceefb4b834db0958d23434f10492f" is_pre_election: true
I20260812 19:03:46.435447 28888 raft_consensus.cc:2454] T f90440cabc2c4d1ebe666503af8ffc10 P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3a45ba920064206b98fbb462d8d4172 in term 0.
I20260812 19:03:46.435415 28456 leader_election.cc:302] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 271fd951ac344d53831f5f23e1decb78, e3a45ba920064206b98fbb462d8d4172; no voters:
I20260812 19:03:46.435727 28984 raft_consensus.cc:2781] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:03:46.435787 28984 raft_consensus.cc:482] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:03:46.435813 28984 raft_consensus.cc:3037] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:46.436842 28984 raft_consensus.cc:504] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:46.437218 28984 leader_election.cc:288] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172 [CANDIDATE]: Term 1 election: Requested vote from peers 271fd951ac344d53831f5f23e1decb78 (127.25.130.2:36689), dcdceefb4b834db0958d23434f10492f (127.25.130.3:42037)
I20260812 19:03:46.437616 28713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f90440cabc2c4d1ebe666503af8ffc10" candidate_uuid: "e3a45ba920064206b98fbb462d8d4172" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "271fd951ac344d53831f5f23e1decb78"
I20260812 19:03:46.437754 28713 raft_consensus.cc:3037] T f90440cabc2c4d1ebe666503af8ffc10 P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:46.437772 28888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f90440cabc2c4d1ebe666503af8ffc10" candidate_uuid: "e3a45ba920064206b98fbb462d8d4172" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcdceefb4b834db0958d23434f10492f"
I20260812 19:03:46.437898 28888 raft_consensus.cc:3037] T f90440cabc2c4d1ebe666503af8ffc10 P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:46.439049 28713 raft_consensus.cc:2454] T f90440cabc2c4d1ebe666503af8ffc10 P 271fd951ac344d53831f5f23e1decb78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3a45ba920064206b98fbb462d8d4172 in term 1.
I20260812 19:03:46.439050 28888 raft_consensus.cc:2454] T f90440cabc2c4d1ebe666503af8ffc10 P dcdceefb4b834db0958d23434f10492f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3a45ba920064206b98fbb462d8d4172 in term 1.
I20260812 19:03:46.439409 28457 leader_election.cc:302] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dcdceefb4b834db0958d23434f10492f, e3a45ba920064206b98fbb462d8d4172; no voters:
I20260812 19:03:46.439649 28984 raft_consensus.cc:2781] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:46.439847 28984 raft_consensus.cc:686] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172 [term 1 LEADER]: Becoming Leader. State: Replica: e3a45ba920064206b98fbb462d8d4172, State: Running, Role: LEADER
I20260812 19:03:46.440128 28984 consensus_queue.cc:237] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:46.442564 28345 catalog_manager.cc:5495] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172 reported cstate change: term changed from 0 to 1, leader changed from <none> to e3a45ba920064206b98fbb462d8d4172 (127.25.130.1). New cstate: current_term: 1 leader_uuid: "e3a45ba920064206b98fbb462d8d4172" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } health_report { overall_health: UNKNOWN } } }
I20260812 19:03:46.480041 28971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:46.476472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41478 (local address 127.25.130.1:36715)
0812 19:03:46.476588 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:46.476592 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:46.476606 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:03:46.476787 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:46.476789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:46.476849 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:46.476940 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:03:46.477229 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.477806 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.478629 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.478903 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.478978 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:46.479200 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:03:46.479727 (+ 527us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:03:46.479836 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:46.479867 (+ 31us) server_negotiation.cc:299] Negotiation successful
0812 19:03:46.479922 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 19:03:46.490751 28968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:46.487404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:40523 (local address 127.25.130.3:42037)
0812 19:03:46.487554 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:46.487558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:46.487645 (+ 87us) server_negotiation.cc:407] Connection header received
0812 19:03:46.487767 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:46.487770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:46.487823 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:46.487886 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:03:46.488137 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.488686 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.489620 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.489864 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.489958 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:46.490356 (+ 398us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:03:46.490439 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:03:46.490506 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:46.490578 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 19:03:46.490633 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 19:03:46.490775 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:46.487265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:42037 (local address 127.25.130.1:40523)
0812 19:03:46.487572 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:46.487585 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:46.487684 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:46.487898 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:46.487900 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:46.487906 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 19:03:46.488017 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:46.488021 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.488816 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.488819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:46.489485 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:46.489495 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.490014 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.490017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:46.490137 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:46.490193 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 19:03:46.490512 (+ 319us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 19:03:46.490517 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 19:03:46.490560 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 19:03:46.490637 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":102,"threads_started":1}
I20260812 19:03:46.497215 28713 raft_consensus.cc:1261] T f90440cabc2c4d1ebe666503af8ffc10 P 271fd951ac344d53831f5f23e1decb78 [term 1 FOLLOWER]: Refusing update from remote peer e3a45ba920064206b98fbb462d8d4172: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:03:46.497529 28888 raft_consensus.cc:1261] T f90440cabc2c4d1ebe666503af8ffc10 P dcdceefb4b834db0958d23434f10492f [term 1 FOLLOWER]: Refusing update from remote peer e3a45ba920064206b98fbb462d8d4172: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:03:46.497704 28984 consensus_queue.cc:1035] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172 [LEADER]: Connected to new peer: Peer: permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:03:46.497989 29036 consensus_queue.cc:1035] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:03:46.518221 28954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:46.514762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:36715 (local address 127.25.130.3:48077)
0812 19:03:46.514888 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:46.514904 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:46.514990 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:46.515260 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:46.515263 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:46.515273 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 19:03:46.515389 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:46.515395 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.516108 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.516111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:46.516801 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:46.516814 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.517520 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.517524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:46.517598 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:46.517630 (+ 32us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 19:03:46.517927 (+ 297us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 19:03:46.517934 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 19:03:46.517994 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 19:03:46.518094 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 19:03:46.518215 28971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:46.514839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:48077 (local address 127.25.130.1:36715)
0812 19:03:46.514948 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:46.514953 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:46.514968 (+ 15us) server_negotiation.cc:407] Connection header received
0812 19:03:46.515079 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:46.515082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:46.515129 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:46.515189 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:03:46.515491 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.516024 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.516950 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:46.517348 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:46.517506 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:46.517764 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:03:46.517848 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:03:46.517911 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:46.518005 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 19:03:46.518061 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260812 19:03:46.519052 29006 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 19:03:48.082647 28344 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:03:48.082837 28344 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:03:48.085783 28344 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5996bc47f2674a18a22d0e59dee1cf32: Sending DeleteTablet for 3 replicas of tablet f90440cabc2c4d1ebe666503af8ffc10
I20260812 19:03:48.086585 28517 tablet_service.cc:1514] Processing DeleteTablet for tablet f90440cabc2c4d1ebe666503af8ffc10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:03:48 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:41476
I20260812 19:03:48.086653 28687 tablet_service.cc:1514] Processing DeleteTablet for tablet f90440cabc2c4d1ebe666503af8ffc10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:03:48 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:44676
I20260812 19:03:48.086654 28861 tablet_service.cc:1514] Processing DeleteTablet for tablet f90440cabc2c4d1ebe666503af8ffc10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:03:48 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:57314
I20260812 19:03:48.087051 29084 tablet_replica.cc:330] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172: stopping tablet replica
I20260812 19:03:48.087302 29084 raft_consensus.cc:2229] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172 [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:03:48.087695 29084 raft_consensus.cc:2258] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:03:48.087983 29085 tablet_replica.cc:330] T f90440cabc2c4d1ebe666503af8ffc10 P 271fd951ac344d53831f5f23e1decb78: stopping tablet replica
I20260812 19:03:48.088383 29082 tablet_replica.cc:330] T f90440cabc2c4d1ebe666503af8ffc10 P dcdceefb4b834db0958d23434f10492f: stopping tablet replica
I20260812 19:03:48.088527 29082 raft_consensus.cc:2229] T f90440cabc2c4d1ebe666503af8ffc10 P dcdceefb4b834db0958d23434f10492f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:03:48.088563 29084 ts_tablet_manager.cc:1905] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:03:48.088666 29082 raft_consensus.cc:2258] T f90440cabc2c4d1ebe666503af8ffc10 P dcdceefb4b834db0958d23434f10492f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:03:48.088872 29085 raft_consensus.cc:2229] T f90440cabc2c4d1ebe666503af8ffc10 P 271fd951ac344d53831f5f23e1decb78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:03:48.089023 29085 raft_consensus.cc:2258] T f90440cabc2c4d1ebe666503af8ffc10 P 271fd951ac344d53831f5f23e1decb78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:03:48.091463 29085 ts_tablet_manager.cc:1905] T f90440cabc2c4d1ebe666503af8ffc10 P 271fd951ac344d53831f5f23e1decb78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:03:48.091562 29082 ts_tablet_manager.cc:1905] T f90440cabc2c4d1ebe666503af8ffc10 P dcdceefb4b834db0958d23434f10492f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:03:48.091692 29084 ts_tablet_manager.cc:1918] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 19:03:48.091779 29084 log.cc:1196] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-0/wal/wals/f90440cabc2c4d1ebe666503af8ffc10
I20260812 19:03:48.093704 29084 ts_tablet_manager.cc:1939] T f90440cabc2c4d1ebe666503af8ffc10 P e3a45ba920064206b98fbb462d8d4172: Deleting consensus metadata
I20260812 19:03:48.094352 29082 ts_tablet_manager.cc:1918] T f90440cabc2c4d1ebe666503af8ffc10 P dcdceefb4b834db0958d23434f10492f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 19:03:48.094362 29085 ts_tablet_manager.cc:1918] T f90440cabc2c4d1ebe666503af8ffc10 P 271fd951ac344d53831f5f23e1decb78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 19:03:48.094446 29085 log.cc:1196] T f90440cabc2c4d1ebe666503af8ffc10 P 271fd951ac344d53831f5f23e1decb78: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-1/wal/wals/f90440cabc2c4d1ebe666503af8ffc10
I20260812 19:03:48.094449 29082 log.cc:1196] T f90440cabc2c4d1ebe666503af8ffc10 P dcdceefb4b834db0958d23434f10492f: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-2/wal/wals/f90440cabc2c4d1ebe666503af8ffc10
I20260812 19:03:48.094715 29082 ts_tablet_manager.cc:1939] T f90440cabc2c4d1ebe666503af8ffc10 P dcdceefb4b834db0958d23434f10492f: Deleting consensus metadata
I20260812 19:03:48.094728 29085 ts_tablet_manager.cc:1939] T f90440cabc2c4d1ebe666503af8ffc10 P 271fd951ac344d53831f5f23e1decb78: Deleting consensus metadata
I20260812 19:03:48.095378 28328 catalog_manager.cc:4841] TS e3a45ba920064206b98fbb462d8d4172 (127.25.130.1:36715): tablet f90440cabc2c4d1ebe666503af8ffc10 (table test-table [id=62692c548c3a4b6abcc99c7179fb915e]) successfully deleted
I20260812 19:03:48.095492 28326 catalog_manager.cc:4841] TS 271fd951ac344d53831f5f23e1decb78 (127.25.130.2:36689): tablet f90440cabc2c4d1ebe666503af8ffc10 (table test-table [id=62692c548c3a4b6abcc99c7179fb915e]) successfully deleted
I20260812 19:03:48.095395 28327 catalog_manager.cc:4841] TS dcdceefb4b834db0958d23434f10492f (127.25.130.3:42037): tablet f90440cabc2c4d1ebe666503af8ffc10 (table test-table [id=62692c548c3a4b6abcc99c7179fb915e]) successfully deleted
Aug 12 19:03:48 dist-test-slave-49h3 krb5kdc[28284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561428, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 19:03:48 dist-test-slave-49h3 krb5kdc[28284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561428, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:48.120157 29094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:48.111387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59078 (local address 127.25.130.62:38757)
0812 19:03:48.111684 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:48.111693 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:48.111711 (+ 18us) server_negotiation.cc:407] Connection header received
0812 19:03:48.111766 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:48.111770 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:48.111819 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:48.111895 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:48.112668 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:48.113234 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:48.114170 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:48.114409 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:48.114521 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:48.116966 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:48.116969 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:48.116972 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:48.117003 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:48.118415 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:48.118871 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:48.118874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:48.118875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:48.118965 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:48.119245 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:48.119248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:48.119250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:48.119456 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:48.119590 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:48.119839 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 19:03:48.119976 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":97,"threads_started":1}
I20260812 19:03:48.122994 28344 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59078:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:03:48.123427 28344 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:03:48.128530 28517 tablet_service.cc:1467] Processing CreateTablet for tablet 1669473c4617449a8b7749c2b5777590 (DEFAULT_TABLE table=test-table [id=4cf9806842a4431384ae307bca9247e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:03:48.128540 28687 tablet_service.cc:1467] Processing CreateTablet for tablet 1669473c4617449a8b7749c2b5777590 (DEFAULT_TABLE table=test-table [id=4cf9806842a4431384ae307bca9247e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:03:48.128669 28861 tablet_service.cc:1467] Processing CreateTablet for tablet 1669473c4617449a8b7749c2b5777590 (DEFAULT_TABLE table=test-table [id=4cf9806842a4431384ae307bca9247e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:03:48.128868 28517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1669473c4617449a8b7749c2b5777590. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:48.128868 28687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1669473c4617449a8b7749c2b5777590. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:48.128979 28861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1669473c4617449a8b7749c2b5777590. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:48.131737 29097 tablet_bootstrap.cc:492] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172: Bootstrap starting.
I20260812 19:03:48.131808 29095 tablet_bootstrap.cc:492] T 1669473c4617449a8b7749c2b5777590 P dcdceefb4b834db0958d23434f10492f: Bootstrap starting.
I20260812 19:03:48.132725 29095 tablet_bootstrap.cc:654] T 1669473c4617449a8b7749c2b5777590 P dcdceefb4b834db0958d23434f10492f: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:48.132725 29097 tablet_bootstrap.cc:654] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:48.133412 29096 tablet_bootstrap.cc:492] T 1669473c4617449a8b7749c2b5777590 P 271fd951ac344d53831f5f23e1decb78: Bootstrap starting.
I20260812 19:03:48.134049 29095 tablet_bootstrap.cc:492] T 1669473c4617449a8b7749c2b5777590 P dcdceefb4b834db0958d23434f10492f: No bootstrap required, opened a new log
I20260812 19:03:48.134114 29097 tablet_bootstrap.cc:492] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172: No bootstrap required, opened a new log
I20260812 19:03:48.134146 29095 ts_tablet_manager.cc:1397] T 1669473c4617449a8b7749c2b5777590 P dcdceefb4b834db0958d23434f10492f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 19:03:48.134210 29097 ts_tablet_manager.cc:1397] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 19:03:48.134352 29096 tablet_bootstrap.cc:654] T 1669473c4617449a8b7749c2b5777590 P 271fd951ac344d53831f5f23e1decb78: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:48.134610 29095 raft_consensus.cc:348] T 1669473c4617449a8b7749c2b5777590 P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:48.134653 29097 raft_consensus.cc:348] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:48.134727 29095 raft_consensus.cc:374] T 1669473c4617449a8b7749c2b5777590 P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:48.134759 29095 raft_consensus.cc:729] T 1669473c4617449a8b7749c2b5777590 P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcdceefb4b834db0958d23434f10492f, State: Initialized, Role: FOLLOWER
I20260812 19:03:48.134763 29097 raft_consensus.cc:374] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:48.134795 29097 raft_consensus.cc:729] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3a45ba920064206b98fbb462d8d4172, State: Initialized, Role: FOLLOWER
I20260812 19:03:48.134919 29095 consensus_queue.cc:260] T 1669473c4617449a8b7749c2b5777590 P dcdceefb4b834db0958d23434f10492f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:48.134950 29097 consensus_queue.cc:260] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:48.135237 29097 ts_tablet_manager.cc:1428] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 19:03:48.135634 29095 ts_tablet_manager.cc:1428] T 1669473c4617449a8b7749c2b5777590 P dcdceefb4b834db0958d23434f10492f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 19:03:48.136075 29096 tablet_bootstrap.cc:492] T 1669473c4617449a8b7749c2b5777590 P 271fd951ac344d53831f5f23e1decb78: No bootstrap required, opened a new log
I20260812 19:03:48.136178 29096 ts_tablet_manager.cc:1397] T 1669473c4617449a8b7749c2b5777590 P 271fd951ac344d53831f5f23e1decb78: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 19:03:48.136675 29096 raft_consensus.cc:348] T 1669473c4617449a8b7749c2b5777590 P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:48.136799 29096 raft_consensus.cc:374] T 1669473c4617449a8b7749c2b5777590 P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:48.136834 29096 raft_consensus.cc:729] T 1669473c4617449a8b7749c2b5777590 P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 271fd951ac344d53831f5f23e1decb78, State: Initialized, Role: FOLLOWER
I20260812 19:03:48.136960 29096 consensus_queue.cc:260] T 1669473c4617449a8b7749c2b5777590 P 271fd951ac344d53831f5f23e1decb78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:48.137392 29096 ts_tablet_manager.cc:1428] T 1669473c4617449a8b7749c2b5777590 P 271fd951ac344d53831f5f23e1decb78: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 19:03:48.182886 29072 raft_consensus.cc:482] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:03:48.183039 29072 raft_consensus.cc:504] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:48.183457 29072 leader_election.cc:288] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 271fd951ac344d53831f5f23e1decb78 (127.25.130.2:36689), dcdceefb4b834db0958d23434f10492f (127.25.130.3:42037)
I20260812 19:03:48.184010 28713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1669473c4617449a8b7749c2b5777590" candidate_uuid: "e3a45ba920064206b98fbb462d8d4172" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "271fd951ac344d53831f5f23e1decb78" is_pre_election: true
I20260812 19:03:48.184010 28888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1669473c4617449a8b7749c2b5777590" candidate_uuid: "e3a45ba920064206b98fbb462d8d4172" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcdceefb4b834db0958d23434f10492f" is_pre_election: true
I20260812 19:03:48.184186 28713 raft_consensus.cc:2454] T 1669473c4617449a8b7749c2b5777590 P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3a45ba920064206b98fbb462d8d4172 in term 0.
I20260812 19:03:48.184199 28888 raft_consensus.cc:2454] T 1669473c4617449a8b7749c2b5777590 P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3a45ba920064206b98fbb462d8d4172 in term 0.
I20260812 19:03:48.184572 28456 leader_election.cc:302] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 271fd951ac344d53831f5f23e1decb78, dcdceefb4b834db0958d23434f10492f, e3a45ba920064206b98fbb462d8d4172; no voters:
I20260812 19:03:48.184804 29072 raft_consensus.cc:2781] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:03:48.184868 29072 raft_consensus.cc:482] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:03:48.184906 29072 raft_consensus.cc:3037] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:48.186120 29072 raft_consensus.cc:504] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:48.186503 29072 leader_election.cc:288] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172 [CANDIDATE]: Term 1 election: Requested vote from peers 271fd951ac344d53831f5f23e1decb78 (127.25.130.2:36689), dcdceefb4b834db0958d23434f10492f (127.25.130.3:42037)
I20260812 19:03:48.187127 28888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1669473c4617449a8b7749c2b5777590" candidate_uuid: "e3a45ba920064206b98fbb462d8d4172" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcdceefb4b834db0958d23434f10492f"
I20260812 19:03:48.187129 28713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1669473c4617449a8b7749c2b5777590" candidate_uuid: "e3a45ba920064206b98fbb462d8d4172" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "271fd951ac344d53831f5f23e1decb78"
I20260812 19:03:48.187280 28888 raft_consensus.cc:3037] T 1669473c4617449a8b7749c2b5777590 P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:48.187300 28713 raft_consensus.cc:3037] T 1669473c4617449a8b7749c2b5777590 P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:48.188530 28888 raft_consensus.cc:2454] T 1669473c4617449a8b7749c2b5777590 P dcdceefb4b834db0958d23434f10492f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3a45ba920064206b98fbb462d8d4172 in term 1.
I20260812 19:03:48.188530 28713 raft_consensus.cc:2454] T 1669473c4617449a8b7749c2b5777590 P 271fd951ac344d53831f5f23e1decb78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3a45ba920064206b98fbb462d8d4172 in term 1.
I20260812 19:03:48.188913 28456 leader_election.cc:302] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 271fd951ac344d53831f5f23e1decb78, e3a45ba920064206b98fbb462d8d4172; no voters:
I20260812 19:03:48.189160 29072 raft_consensus.cc:2781] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:48.189261 29072 raft_consensus.cc:686] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172 [term 1 LEADER]: Becoming Leader. State: Replica: e3a45ba920064206b98fbb462d8d4172, State: Running, Role: LEADER
I20260812 19:03:48.189411 29072 consensus_queue.cc:237] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:48.191396 28345 catalog_manager.cc:5495] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172 reported cstate change: term changed from 0 to 1, leader changed from <none> to e3a45ba920064206b98fbb462d8d4172 (127.25.130.1). New cstate: current_term: 1 leader_uuid: "e3a45ba920064206b98fbb462d8d4172" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } health_report { overall_health: UNKNOWN } } }
I20260812 19:03:48.207049 29106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:48.203946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41488 (local address 127.25.130.1:36715)
0812 19:03:48.204214 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:48.204220 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:48.204232 (+ 12us) server_negotiation.cc:407] Connection header received
0812 19:03:48.204276 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:48.204278 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:48.204322 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:48.204390 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:03:48.204695 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:48.205232 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:48.206164 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:48.206399 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:48.206470 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:48.206685 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:03:48.206758 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:03:48.206803 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:48.206872 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 19:03:48.206919 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":103,"threads_started":1}
I20260812 19:03:48.213217 28713 raft_consensus.cc:1261] T 1669473c4617449a8b7749c2b5777590 P 271fd951ac344d53831f5f23e1decb78 [term 1 FOLLOWER]: Refusing update from remote peer e3a45ba920064206b98fbb462d8d4172: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:03:48.213469 28888 raft_consensus.cc:1261] T 1669473c4617449a8b7749c2b5777590 P dcdceefb4b834db0958d23434f10492f [term 1 FOLLOWER]: Refusing update from remote peer e3a45ba920064206b98fbb462d8d4172: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:03:48.213629 29072 consensus_queue.cc:1035] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172 [LEADER]: Connected to new peer: Peer: permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:03:48.213837 29075 consensus_queue.cc:1035] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:03:48.244870 28345 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:03:48.245038 28345 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:03:48.247401 28345 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5996bc47f2674a18a22d0e59dee1cf32: Sending DeleteTablet for 3 replicas of tablet 1669473c4617449a8b7749c2b5777590
I20260812 19:03:48.248055 28517 tablet_service.cc:1514] Processing DeleteTablet for tablet 1669473c4617449a8b7749c2b5777590 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:03:48 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:41476
I20260812 19:03:48.248085 28687 tablet_service.cc:1514] Processing DeleteTablet for tablet 1669473c4617449a8b7749c2b5777590 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:03:48 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:44676
I20260812 19:03:48.248304 29085 tablet_replica.cc:330] T 1669473c4617449a8b7749c2b5777590 P 271fd951ac344d53831f5f23e1decb78: stopping tablet replica
I20260812 19:03:48.248304 29084 tablet_replica.cc:330] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172: stopping tablet replica
I20260812 19:03:48.248438 29085 raft_consensus.cc:2229] T 1669473c4617449a8b7749c2b5777590 P 271fd951ac344d53831f5f23e1decb78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:03:48.248435 29084 raft_consensus.cc:2229] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172 [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:03:48.248569 29085 raft_consensus.cc:2258] T 1669473c4617449a8b7749c2b5777590 P 271fd951ac344d53831f5f23e1decb78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:03:48.248642 29084 raft_consensus.cc:2258] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:03:48.248953 28861 tablet_service.cc:1514] Processing DeleteTablet for tablet 1669473c4617449a8b7749c2b5777590 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:03:48 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:57314
I20260812 19:03:48.249190 29082 tablet_replica.cc:330] T 1669473c4617449a8b7749c2b5777590 P dcdceefb4b834db0958d23434f10492f: stopping tablet replica
I20260812 19:03:48.249167 28345 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59078:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 19:03:48.249315 29082 raft_consensus.cc:2229] T 1669473c4617449a8b7749c2b5777590 P dcdceefb4b834db0958d23434f10492f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:03:48.249387 29084 ts_tablet_manager.cc:1905] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:03:48.249426 29082 raft_consensus.cc:2258] T 1669473c4617449a8b7749c2b5777590 P dcdceefb4b834db0958d23434f10492f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:03:48.249641 29085 ts_tablet_manager.cc:1905] T 1669473c4617449a8b7749c2b5777590 P 271fd951ac344d53831f5f23e1decb78: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 19:03:48.249665 28345 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:03:48.252187 29084 ts_tablet_manager.cc:1918] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 19:03:48.252187 29085 ts_tablet_manager.cc:1918] T 1669473c4617449a8b7749c2b5777590 P 271fd951ac344d53831f5f23e1decb78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 19:03:48.252269 29084 log.cc:1196] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-0/wal/wals/1669473c4617449a8b7749c2b5777590
I20260812 19:03:48.252269 29085 log.cc:1196] T 1669473c4617449a8b7749c2b5777590 P 271fd951ac344d53831f5f23e1decb78: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-1/wal/wals/1669473c4617449a8b7749c2b5777590
I20260812 19:03:48.252522 29084 ts_tablet_manager.cc:1939] T 1669473c4617449a8b7749c2b5777590 P e3a45ba920064206b98fbb462d8d4172: Deleting consensus metadata
I20260812 19:03:48.252522 29085 ts_tablet_manager.cc:1939] T 1669473c4617449a8b7749c2b5777590 P 271fd951ac344d53831f5f23e1decb78: Deleting consensus metadata
I20260812 19:03:48.253480 28326 catalog_manager.cc:4841] TS 271fd951ac344d53831f5f23e1decb78 (127.25.130.2:36689): tablet 1669473c4617449a8b7749c2b5777590 (table test-table [id=4cf9806842a4431384ae307bca9247e2]) successfully deleted
I20260812 19:03:48.253732 29082 ts_tablet_manager.cc:1905] T 1669473c4617449a8b7749c2b5777590 P dcdceefb4b834db0958d23434f10492f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:03:48.254750 28517 tablet_service.cc:1467] Processing CreateTablet for tablet a6c88b0ead93468fa78ad491a41015d1 (DEFAULT_TABLE table=test-table [id=2e57d3eb9acb442cb2fba1535c47a07c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:03:48.255033 28517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6c88b0ead93468fa78ad491a41015d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:48.255026 28687 tablet_service.cc:1467] Processing CreateTablet for tablet a6c88b0ead93468fa78ad491a41015d1 (DEFAULT_TABLE table=test-table [id=2e57d3eb9acb442cb2fba1535c47a07c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:03:48.255260 28687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6c88b0ead93468fa78ad491a41015d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:48.256366 29082 ts_tablet_manager.cc:1918] T 1669473c4617449a8b7749c2b5777590 P dcdceefb4b834db0958d23434f10492f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 19:03:48.256428 29082 log.cc:1196] T 1669473c4617449a8b7749c2b5777590 P dcdceefb4b834db0958d23434f10492f: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-2/wal/wals/1669473c4617449a8b7749c2b5777590
I20260812 19:03:48.256599 29082 ts_tablet_manager.cc:1939] T 1669473c4617449a8b7749c2b5777590 P dcdceefb4b834db0958d23434f10492f: Deleting consensus metadata
I20260812 19:03:48.257419 29097 tablet_bootstrap.cc:492] T a6c88b0ead93468fa78ad491a41015d1 P e3a45ba920064206b98fbb462d8d4172: Bootstrap starting.
I20260812 19:03:48.257419 29096 tablet_bootstrap.cc:492] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78: Bootstrap starting.
I20260812 19:03:48.258371 29097 tablet_bootstrap.cc:654] T a6c88b0ead93468fa78ad491a41015d1 P e3a45ba920064206b98fbb462d8d4172: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:48.258371 29096 tablet_bootstrap.cc:654] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:48.259068 28328 catalog_manager.cc:4841] TS e3a45ba920064206b98fbb462d8d4172 (127.25.130.1:36715): tablet 1669473c4617449a8b7749c2b5777590 (table test-table [id=4cf9806842a4431384ae307bca9247e2]) successfully deleted
I20260812 19:03:48.259265 28327 catalog_manager.cc:4841] TS dcdceefb4b834db0958d23434f10492f (127.25.130.3:42037): tablet 1669473c4617449a8b7749c2b5777590 (table test-table [id=4cf9806842a4431384ae307bca9247e2]) successfully deleted
I20260812 19:03:48.259687 28861 tablet_service.cc:1467] Processing CreateTablet for tablet a6c88b0ead93468fa78ad491a41015d1 (DEFAULT_TABLE table=test-table [id=2e57d3eb9acb442cb2fba1535c47a07c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:03:48.259872 29097 tablet_bootstrap.cc:492] T a6c88b0ead93468fa78ad491a41015d1 P e3a45ba920064206b98fbb462d8d4172: No bootstrap required, opened a new log
I20260812 19:03:48.259958 29097 ts_tablet_manager.cc:1397] T a6c88b0ead93468fa78ad491a41015d1 P e3a45ba920064206b98fbb462d8d4172: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 19:03:48.259980 28861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6c88b0ead93468fa78ad491a41015d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:48.260401 29097 raft_consensus.cc:348] T a6c88b0ead93468fa78ad491a41015d1 P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:48.260511 29097 raft_consensus.cc:374] T a6c88b0ead93468fa78ad491a41015d1 P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:48.260537 29097 raft_consensus.cc:729] T a6c88b0ead93468fa78ad491a41015d1 P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3a45ba920064206b98fbb462d8d4172, State: Initialized, Role: FOLLOWER
I20260812 19:03:48.260659 29097 consensus_queue.cc:260] T a6c88b0ead93468fa78ad491a41015d1 P e3a45ba920064206b98fbb462d8d4172 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:48.260903 29097 ts_tablet_manager.cc:1428] T a6c88b0ead93468fa78ad491a41015d1 P e3a45ba920064206b98fbb462d8d4172: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 19:03:48.262203 29096 tablet_bootstrap.cc:492] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78: No bootstrap required, opened a new log
I20260812 19:03:48.262298 29096 ts_tablet_manager.cc:1397] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 19:03:48.262298 29095 tablet_bootstrap.cc:492] T a6c88b0ead93468fa78ad491a41015d1 P dcdceefb4b834db0958d23434f10492f: Bootstrap starting.
I20260812 19:03:48.262705 29096 raft_consensus.cc:348] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:48.262806 29096 raft_consensus.cc:374] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:48.262835 29096 raft_consensus.cc:729] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 271fd951ac344d53831f5f23e1decb78, State: Initialized, Role: FOLLOWER
I20260812 19:03:48.262969 29096 consensus_queue.cc:260] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:48.263043 29095 tablet_bootstrap.cc:654] T a6c88b0ead93468fa78ad491a41015d1 P dcdceefb4b834db0958d23434f10492f: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:48.263223 29096 ts_tablet_manager.cc:1428] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 19:03:48.264150 29095 tablet_bootstrap.cc:492] T a6c88b0ead93468fa78ad491a41015d1 P dcdceefb4b834db0958d23434f10492f: No bootstrap required, opened a new log
I20260812 19:03:48.264261 29095 ts_tablet_manager.cc:1397] T a6c88b0ead93468fa78ad491a41015d1 P dcdceefb4b834db0958d23434f10492f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 19:03:48.264685 29095 raft_consensus.cc:348] T a6c88b0ead93468fa78ad491a41015d1 P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:48.264832 29095 raft_consensus.cc:374] T a6c88b0ead93468fa78ad491a41015d1 P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:48.264875 29095 raft_consensus.cc:729] T a6c88b0ead93468fa78ad491a41015d1 P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcdceefb4b834db0958d23434f10492f, State: Initialized, Role: FOLLOWER
I20260812 19:03:48.265008 29095 consensus_queue.cc:260] T a6c88b0ead93468fa78ad491a41015d1 P dcdceefb4b834db0958d23434f10492f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:48.265275 29095 ts_tablet_manager.cc:1428] T a6c88b0ead93468fa78ad491a41015d1 P dcdceefb4b834db0958d23434f10492f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 19:03:48.299265 29102 raft_consensus.cc:482] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:03:48.299427 29102 raft_consensus.cc:504] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:48.299824 29102 leader_election.cc:288] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3a45ba920064206b98fbb462d8d4172 (127.25.130.1:36715), dcdceefb4b834db0958d23434f10492f (127.25.130.3:42037)
I20260812 19:03:48.300333 28888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6c88b0ead93468fa78ad491a41015d1" candidate_uuid: "271fd951ac344d53831f5f23e1decb78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcdceefb4b834db0958d23434f10492f" is_pre_election: true
I20260812 19:03:48.300356 28544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6c88b0ead93468fa78ad491a41015d1" candidate_uuid: "271fd951ac344d53831f5f23e1decb78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3a45ba920064206b98fbb462d8d4172" is_pre_election: true
I20260812 19:03:48.300493 28888 raft_consensus.cc:2454] T a6c88b0ead93468fa78ad491a41015d1 P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 271fd951ac344d53831f5f23e1decb78 in term 0.
I20260812 19:03:48.300524 28544 raft_consensus.cc:2454] T a6c88b0ead93468fa78ad491a41015d1 P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 271fd951ac344d53831f5f23e1decb78 in term 0.
I20260812 19:03:48.300869 28629 leader_election.cc:302] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 271fd951ac344d53831f5f23e1decb78, dcdceefb4b834db0958d23434f10492f; no voters:
I20260812 19:03:48.301072 29102 raft_consensus.cc:2781] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:03:48.301136 29102 raft_consensus.cc:482] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:03:48.301182 29102 raft_consensus.cc:3037] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:48.302301 29102 raft_consensus.cc:504] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:48.302675 29102 leader_election.cc:288] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78 [CANDIDATE]: Term 1 election: Requested vote from peers e3a45ba920064206b98fbb462d8d4172 (127.25.130.1:36715), dcdceefb4b834db0958d23434f10492f (127.25.130.3:42037)
I20260812 19:03:48.303069 28888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6c88b0ead93468fa78ad491a41015d1" candidate_uuid: "271fd951ac344d53831f5f23e1decb78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcdceefb4b834db0958d23434f10492f"
I20260812 19:03:48.303071 28544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6c88b0ead93468fa78ad491a41015d1" candidate_uuid: "271fd951ac344d53831f5f23e1decb78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3a45ba920064206b98fbb462d8d4172"
I20260812 19:03:48.303185 28888 raft_consensus.cc:3037] T a6c88b0ead93468fa78ad491a41015d1 P dcdceefb4b834db0958d23434f10492f [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:48.303195 28544 raft_consensus.cc:3037] T a6c88b0ead93468fa78ad491a41015d1 P e3a45ba920064206b98fbb462d8d4172 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:48.304136 28888 raft_consensus.cc:2454] T a6c88b0ead93468fa78ad491a41015d1 P dcdceefb4b834db0958d23434f10492f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 271fd951ac344d53831f5f23e1decb78 in term 1.
I20260812 19:03:48.304193 28544 raft_consensus.cc:2454] T a6c88b0ead93468fa78ad491a41015d1 P e3a45ba920064206b98fbb462d8d4172 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 271fd951ac344d53831f5f23e1decb78 in term 1.
I20260812 19:03:48.304471 28629 leader_election.cc:302] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 271fd951ac344d53831f5f23e1decb78, dcdceefb4b834db0958d23434f10492f; no voters:
I20260812 19:03:48.304728 29102 raft_consensus.cc:2781] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:48.304914 29102 raft_consensus.cc:686] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78 [term 1 LEADER]: Becoming Leader. State: Replica: 271fd951ac344d53831f5f23e1decb78, State: Running, Role: LEADER
I20260812 19:03:48.305164 29102 consensus_queue.cc:237] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } }
I20260812 19:03:48.307619 28344 catalog_manager.cc:5495] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 271fd951ac344d53831f5f23e1decb78 (127.25.130.2). New cstate: current_term: 1 leader_uuid: "271fd951ac344d53831f5f23e1decb78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "271fd951ac344d53831f5f23e1decb78" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 36689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 } health_report { overall_health: UNKNOWN } } }
I20260812 19:03:48.334275 29117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:48.330608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44678 (local address 127.25.130.2:36689)
0812 19:03:48.330948 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:48.330955 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:48.330967 (+ 12us) server_negotiation.cc:407] Connection header received
0812 19:03:48.331051 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:48.331054 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:48.331113 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:48.331219 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:03:48.331407 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:48.331947 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:48.332757 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:48.333025 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:48.333111 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:48.333336 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:03:48.333900 (+ 564us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:03:48.333994 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:48.334041 (+ 47us) server_negotiation.cc:299] Negotiation successful
0812 19:03:48.334096 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":173,"threads_started":1}
I20260812 19:03:48.345839 29120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:48.341770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:42037 (local address 127.25.130.2:41721)
0812 19:03:48.342118 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:48.342130 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:48.342224 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:48.342476 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:48.342478 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:48.342483 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 19:03:48.343073 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:48.343082 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:48.343950 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:48.343952 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:48.344593 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:48.344603 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:48.345098 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:48.345101 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:48.345184 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:48.345239 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 19:03:48.345544 (+ 305us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 19:03:48.345550 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 19:03:48.345606 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 19:03:48.345685 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":121,"threads_started":1}
I20260812 19:03:48.345829 29122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:48.342020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:41721 (local address 127.25.130.3:42037)
0812 19:03:48.342269 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:48.342281 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:48.342294 (+ 13us) server_negotiation.cc:407] Connection header received
0812 19:03:48.342339 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:48.342341 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:48.342387 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:48.342459 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:03:48.343224 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:48.343846 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:48.344733 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:48.344969 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:48.345051 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:48.345357 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:03:48.345439 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:03:48.345518 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:48.345626 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 19:03:48.345680 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":80,"threads_started":1}
I20260812 19:03:48.352087 28544 raft_consensus.cc:1261] T a6c88b0ead93468fa78ad491a41015d1 P e3a45ba920064206b98fbb462d8d4172 [term 1 FOLLOWER]: Refusing update from remote peer 271fd951ac344d53831f5f23e1decb78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:03:48.352087 28888 raft_consensus.cc:1261] T a6c88b0ead93468fa78ad491a41015d1 P dcdceefb4b834db0958d23434f10492f [term 1 FOLLOWER]: Refusing update from remote peer 271fd951ac344d53831f5f23e1decb78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:03:48.352596 29102 consensus_queue.cc:1035] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3a45ba920064206b98fbb462d8d4172" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:03:48.352834 29114 consensus_queue.cc:1035] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcdceefb4b834db0958d23434f10492f" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 42037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:03:48.371166 29129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:48.367564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:36689 (local address 127.25.130.3:40171)
0812 19:03:48.367866 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:48.367880 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:48.367970 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:48.368345 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:48.368347 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:48.368354 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 19:03:48.368467 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:48.368474 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:48.369312 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:48.369315 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:48.369988 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:48.369997 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:48.370554 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:48.370556 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:48.370630 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:48.370662 (+ 32us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 19:03:48.370913 (+ 251us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 19:03:48.370917 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 19:03:48.370958 (+ 41us) client_negotiation.cc:240] Negotiation successful
0812 19:03:48.371026 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":112,"threads_started":1}
I20260812 19:03:48.371155 29117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:48.367682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:40171 (local address 127.25.130.2:36689)
0812 19:03:48.367805 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:48.367810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:48.367938 (+ 128us) server_negotiation.cc:407] Connection header received
0812 19:03:48.368043 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:48.368046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:48.368101 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:48.368177 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:03:48.368584 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:48.369192 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:48.370095 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:48.370455 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:48.370557 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:48.370763 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:03:48.370847 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:03:48.370909 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:48.370967 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 19:03:48.371021 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260812 19:03:48.371969 29059 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 19:03:49.940253 28345 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:03:49.940510 28345 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:03:49.945761 28345 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5996bc47f2674a18a22d0e59dee1cf32: Sending DeleteTablet for 3 replicas of tablet a6c88b0ead93468fa78ad491a41015d1
I20260812 19:03:49.946485 28687 tablet_service.cc:1514] Processing DeleteTablet for tablet a6c88b0ead93468fa78ad491a41015d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:03:49 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:44676
I20260812 19:03:49.946588 28861 tablet_service.cc:1514] Processing DeleteTablet for tablet a6c88b0ead93468fa78ad491a41015d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:03:49 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:57314
I20260812 19:03:49.946925 28517 tablet_service.cc:1514] Processing DeleteTablet for tablet a6c88b0ead93468fa78ad491a41015d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:03:49 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:41476
I20260812 19:03:49.946961 29158 tablet_replica.cc:330] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78: stopping tablet replica
I20260812 19:03:49.947036 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 28436
I20260812 19:03:49.947131 29158 raft_consensus.cc:2229] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78 [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:03:49.957227 29158 raft_consensus.cc:2258] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:03:49.958223 29158 ts_tablet_manager.cc:1905] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:03:49.958545 29159 tablet_replica.cc:330] T a6c88b0ead93468fa78ad491a41015d1 P dcdceefb4b834db0958d23434f10492f: stopping tablet replica
I20260812 19:03:49.958729 29159 raft_consensus.cc:2229] T a6c88b0ead93468fa78ad491a41015d1 P dcdceefb4b834db0958d23434f10492f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:03:49.958909 29159 raft_consensus.cc:2258] T a6c88b0ead93468fa78ad491a41015d1 P dcdceefb4b834db0958d23434f10492f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:03:49.959672 29159 ts_tablet_manager.cc:1905] T a6c88b0ead93468fa78ad491a41015d1 P dcdceefb4b834db0958d23434f10492f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:03:49.961836 29158 ts_tablet_manager.cc:1918] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 19:03:49.961938 29158 log.cc:1196] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-1/wal/wals/a6c88b0ead93468fa78ad491a41015d1
I20260812 19:03:49.962276 29158 ts_tablet_manager.cc:1939] T a6c88b0ead93468fa78ad491a41015d1 P 271fd951ac344d53831f5f23e1decb78: Deleting consensus metadata
I20260812 19:03:49.962910 29159 ts_tablet_manager.cc:1918] T a6c88b0ead93468fa78ad491a41015d1 P dcdceefb4b834db0958d23434f10492f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 19:03:49.963001 29159 log.cc:1196] T a6c88b0ead93468fa78ad491a41015d1 P dcdceefb4b834db0958d23434f10492f: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561415988404-26120-0/minicluster-data/ts-2/wal/wals/a6c88b0ead93468fa78ad491a41015d1
I20260812 19:03:49.963240 29159 ts_tablet_manager.cc:1939] T a6c88b0ead93468fa78ad491a41015d1 P dcdceefb4b834db0958d23434f10492f: Deleting consensus metadata
I20260812 19:03:49.963544 28326 catalog_manager.cc:4841] TS 271fd951ac344d53831f5f23e1decb78 (127.25.130.2:36689): tablet a6c88b0ead93468fa78ad491a41015d1 (table test-table [id=2e57d3eb9acb442cb2fba1535c47a07c]) successfully deleted
I20260812 19:03:49.964042 28327 catalog_manager.cc:4841] TS dcdceefb4b834db0958d23434f10492f (127.25.130.3:42037): tablet a6c88b0ead93468fa78ad491a41015d1 (table test-table [id=2e57d3eb9acb442cb2fba1535c47a07c]) successfully deleted
W20260812 19:03:49.965797 28328 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.130.1:36715 (error 108)
I20260812 19:03:49.966069 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 28607
I20260812 19:03:49.970626 29161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:49.970056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:36715 (local address 127.0.0.1:41492)
0812 19:03:49.970389 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:49.970492 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.1:36715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":112,"threads_started":1}
W20260812 19:03:49.970773 28328 catalog_manager.cc:4584] TS e3a45ba920064206b98fbb462d8d4172 (127.25.130.1:36715): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a6c88b0ead93468fa78ad491a41015d1: Network error: Client connection negotiation failed: client connection to 127.25.130.1:36715: connect: Connection refused (error 111)
I20260812 19:03:49.979607 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 28781
I20260812 19:03:49.995615 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 28302
2026-08-12T19:03:50Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6378 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:03:50 dist-test-slave-49h3 krb5kdc[29165](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:03:50 dist-test-slave-49h3 krb5kdc[29165](info): set up 2 sockets
Aug 12 19:03:50 dist-test-slave-49h3 krb5kdc[29165](info): commencing operation
krb5kdc: starting...
W20260812 19:03:51.089110 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.005s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:03:51 dist-test-slave-49h3 krb5kdc[29165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:03:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:03:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:03:51.224889 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:44945
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:46411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:44945
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:51.342427 29180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:51.342752 29180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:51.342849 29180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:51.347116 29180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:51.347203 29180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:51.347231 29180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:51.347254 29180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:51.347275 29180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:51.352988 29180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:46411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:44945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:44945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:51.354202 29180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:51.355297 29180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:03:51.364490 29180 server_base.cc:1034] running on GCE node
W20260812 19:03:51.364490 29187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:51.364513 29189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:51.364540 29186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:51.365322 29180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:51.366554 29180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:51.367753 29180 hybrid_clock.cc:648] HybridClock initialized: now 1786561431367739 us; error 31 us; skew 500 ppm
Aug 12 19:03:51 dist-test-slave-49h3 krb5kdc[29165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561431, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:51.374622 29180 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:03:51.375662 29180 webserver.cc:460] Webserver started at http://127.25.130.62:43281/ using document root <none> and password file <none>
I20260812 19:03:51.376227 29180 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:51.376291 29180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:51.376472 29180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:51.378702 29180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "fd6d59a9db294cfa81d75eb457f4cd4b"
format_stamp: "Formatted at 2026-08-12 19:03:51 on dist-test-slave-49h3"
server_key: "46d0f5df68181baaf9818c59413cd464"
server_key_iv: "b24c8d1ac53ae3fa3110147ed2b9e9ad"
server_key_version: "encryptionkey@0"
I20260812 19:03:51.379249 29180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "fd6d59a9db294cfa81d75eb457f4cd4b"
format_stamp: "Formatted at 2026-08-12 19:03:51 on dist-test-slave-49h3"
server_key: "46d0f5df68181baaf9818c59413cd464"
server_key_iv: "b24c8d1ac53ae3fa3110147ed2b9e9ad"
server_key_version: "encryptionkey@0"
I20260812 19:03:51.383222 29180 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:03:51.385948 29197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:51.387337 29180 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:03:51.387461 29180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "fd6d59a9db294cfa81d75eb457f4cd4b"
format_stamp: "Formatted at 2026-08-12 19:03:51 on dist-test-slave-49h3"
server_key: "46d0f5df68181baaf9818c59413cd464"
server_key_iv: "b24c8d1ac53ae3fa3110147ed2b9e9ad"
server_key_version: "encryptionkey@0"
I20260812 19:03:51.387570 29180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:51.403182 29180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:51.406467 29180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:51.406652 29180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:51.414366 29180 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:44945
I20260812 19:03:51.414392 29266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:44945 every 8 connection(s)
I20260812 19:03:51.415367 29180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:51.418311 29267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:51.422921 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 29180
I20260812 19:03:51.423059 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:03:51.423322 26120 external_mini_cluster.cc:1428] Setting key 6cfadff542323180d3aba6736b16fe4e
I20260812 19:03:51.424803 29267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd6d59a9db294cfa81d75eb457f4cd4b: Bootstrap starting.
I20260812 19:03:51.427448 29267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd6d59a9db294cfa81d75eb457f4cd4b: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:51.428447 29267 log.cc:824] T 00000000000000000000000000000000 P fd6d59a9db294cfa81d75eb457f4cd4b: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:51.431180 29267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd6d59a9db294cfa81d75eb457f4cd4b: No bootstrap required, opened a new log
Aug 12 19:03:51 dist-test-slave-49h3 krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:51.436877 29267 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd6d59a9db294cfa81d75eb457f4cd4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd6d59a9db294cfa81d75eb457f4cd4b" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 44945 } }
I20260812 19:03:51.437180 29267 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd6d59a9db294cfa81d75eb457f4cd4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:51.437237 29267 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd6d59a9db294cfa81d75eb457f4cd4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd6d59a9db294cfa81d75eb457f4cd4b, State: Initialized, Role: FOLLOWER
I20260812 19:03:51.437808 29267 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd6d59a9db294cfa81d75eb457f4cd4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd6d59a9db294cfa81d75eb457f4cd4b" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 44945 } }
I20260812 19:03:51.437992 29267 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd6d59a9db294cfa81d75eb457f4cd4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:03:51.438068 29267 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd6d59a9db294cfa81d75eb457f4cd4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:03:51.438195 29267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd6d59a9db294cfa81d75eb457f4cd4b [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:51.439378 29267 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd6d59a9db294cfa81d75eb457f4cd4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd6d59a9db294cfa81d75eb457f4cd4b" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 44945 } }
I20260812 19:03:51.439785 29267 leader_election.cc:302] T 00000000000000000000000000000000 P fd6d59a9db294cfa81d75eb457f4cd4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd6d59a9db294cfa81d75eb457f4cd4b; no voters:
I20260812 19:03:51.440227 29267 leader_election.cc:288] T 00000000000000000000000000000000 P fd6d59a9db294cfa81d75eb457f4cd4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:03:51.440383 29272 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd6d59a9db294cfa81d75eb457f4cd4b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:51.440599 29272 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd6d59a9db294cfa81d75eb457f4cd4b [term 1 LEADER]: Becoming Leader. State: Replica: fd6d59a9db294cfa81d75eb457f4cd4b, State: Running, Role: LEADER
I20260812 19:03:51.440928 29272 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd6d59a9db294cfa81d75eb457f4cd4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd6d59a9db294cfa81d75eb457f4cd4b" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 44945 } }
I20260812 19:03:51.441495 29270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:51.425034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39894 (local address 127.25.130.62:44945)
0812 19:03:51.425557 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:51.425573 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:51.425618 (+ 45us) server_negotiation.cc:407] Connection header received
0812 19:03:51.426332 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:51.426351 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:51.426739 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:51.427027 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:51.428057 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:51.429561 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:51.430821 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:51.431616 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:51.431787 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:51.435308 (+ 3521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:51.435314 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:51.435331 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:51.435368 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:51.437678 (+ 2310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:51.438419 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:51.438426 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:51.438434 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:51.438595 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:51.438916 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:51.438918 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:51.438919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:51.439412 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:51.439612 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:51.440012 (+ 400us) server_negotiation.cc:299] Negotiation successful
0812 19:03:51.440385 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":132,"threads_started":1}
I20260812 19:03:51.441668 29267 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd6d59a9db294cfa81d75eb457f4cd4b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:51.442270 29272 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd6d59a9db294cfa81d75eb457f4cd4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd6d59a9db294cfa81d75eb457f4cd4b. Latest consensus state: current_term: 1 leader_uuid: "fd6d59a9db294cfa81d75eb457f4cd4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd6d59a9db294cfa81d75eb457f4cd4b" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 44945 } } }
I20260812 19:03:51.442389 29272 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd6d59a9db294cfa81d75eb457f4cd4b [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:51.444329 29273 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd6d59a9db294cfa81d75eb457f4cd4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd6d59a9db294cfa81d75eb457f4cd4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd6d59a9db294cfa81d75eb457f4cd4b" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 44945 } } }
I20260812 19:03:51.444499 29273 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd6d59a9db294cfa81d75eb457f4cd4b [sys.catalog]: This master's current role is: LEADER
W20260812 19:03:51.447484 29291 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fd6d59a9db294cfa81d75eb457f4cd4b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:03:51.447582 29291 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:03:51.447679 29278 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:51.448559 29278 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:03:51.454916 29278 catalog_manager.cc:1347] Generated new cluster ID: bb55cd2a981f40989b4a9c5f3c1f2c80
I20260812 19:03:51.455003 29278 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:51.461666 29278 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:51.462750 29278 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:51.477068 29278 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd6d59a9db294cfa81d75eb457f4cd4b: Generated new TSK 0
I20260812 19:03:51.477906 29278 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
I20260812 19:03:51.522066 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:0
--local_ip_for_outbound_sockets=127.25.130.1
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:44945
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:46411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:51.667645 29298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:51.667973 29298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:51.668038 29298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:51.672343 29298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:51.672437 29298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:51.672524 29298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:03:51.678359 29298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:46411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:44945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:51.679576 29298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:51.680667 29298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:51.691200 29305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:51.691202 29308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:51.691416 29298 server_base.cc:1034] running on GCE node
W20260812 19:03:51.691840 29304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:51.692364 29298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:51.693259 29298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:51.696529 29298 hybrid_clock.cc:648] HybridClock initialized: now 1786561431696439 us; error 94 us; skew 500 ppm
Aug 12 19:03:51 dist-test-slave-49h3 krb5kdc[29165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561431, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:51.699604 29298 init.cc:377] Logged in from keytab as kudu/127.25.130.1@KRBTEST.COM (short username kudu)
I20260812 19:03:51.700599 29298 webserver.cc:460] Webserver started at http://127.25.130.1:33907/ using document root <none> and password file <none>
I20260812 19:03:51.701187 29298 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:51.701253 29298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:51.701478 29298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:51.703831 29298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/data/instance:
uuid: "d1d30b056f94452eb18e93204778220b"
format_stamp: "Formatted at 2026-08-12 19:03:51 on dist-test-slave-49h3"
server_key: "bed8e275f4b5254689b8f062076a186d"
server_key_iv: "74a3c957e8083d17be14e0fc65e3dc21"
server_key_version: "encryptionkey@0"
I20260812 19:03:51.704463 29298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance:
uuid: "d1d30b056f94452eb18e93204778220b"
format_stamp: "Formatted at 2026-08-12 19:03:51 on dist-test-slave-49h3"
server_key: "bed8e275f4b5254689b8f062076a186d"
server_key_iv: "74a3c957e8083d17be14e0fc65e3dc21"
server_key_version: "encryptionkey@0"
I20260812 19:03:51.708810 29298 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 19:03:51.711628 29317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:51.712989 29298 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 19:03:51.713119 29298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "d1d30b056f94452eb18e93204778220b"
format_stamp: "Formatted at 2026-08-12 19:03:51 on dist-test-slave-49h3"
server_key: "bed8e275f4b5254689b8f062076a186d"
server_key_iv: "74a3c957e8083d17be14e0fc65e3dc21"
server_key_version: "encryptionkey@0"
I20260812 19:03:51.713265 29298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:51.733577 29298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:51.736884 29298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:51.737076 29298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:51.737797 29298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:03:51.738706 29298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:03:51.738757 29298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:51.738801 29298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:03:51.738828 29298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:51.748369 29298 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:36105
I20260812 19:03:51.748395 29460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:36105 every 8 connection(s)
I20260812 19:03:51.749339 29298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:03:51.753069 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 29298
I20260812 19:03:51.753189 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance
I20260812 19:03:51.753479 26120 external_mini_cluster.cc:1428] Setting key 94f2c85fde9f0f6ca392da482d403247
Aug 12 19:03:51 dist-test-slave-49h3 krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561431, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.130.2@KRBTEST.COM; defaulting to no policy
I20260812 19:03:51.764444 29270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:51.751331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:58065 (local address 127.25.130.62:44945)
0812 19:03:51.751526 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:51.751531 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:51.752463 (+ 932us) server_negotiation.cc:407] Connection header received
0812 19:03:51.753881 (+ 1418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:51.753884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:51.753935 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:51.754008 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:51.755344 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:51.756051 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:51.757440 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:51.757726 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:51.757813 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:51.760706 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:51.760714 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:51.760717 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:51.760745 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:51.762164 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:51.762741 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:51.762744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:51.762746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:51.762834 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:51.763200 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:51.763204 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:51.763205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:51.763404 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:51.763493 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:51.764109 (+ 616us) server_negotiation.cc:299] Negotiation successful
0812 19:03:51.764261 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 19:03:51.765429 29464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:51.751764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:44945 (local address 127.25.130.1:58065)
0812 19:03:51.752278 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:51.752327 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:51.753649 (+ 1322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:51.754208 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:51.754215 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:51.754591 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:51.755177 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:51.755192 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:51.756179 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:51.756182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:51.757299 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:51.757312 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:51.757901 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:51.757905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:51.758123 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:51.758776 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:51.758797 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:51.760501 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:51.762303 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:51.762311 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:51.762330 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:51.762617 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:51.762931 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:51.762934 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:51.762937 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:51.763098 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:51.763524 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:51.763532 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:51.763849 (+ 317us) client_negotiation.cc:769] Sending connection context
0812 19:03:51.764038 (+ 189us) client_negotiation.cc:240] Negotiation successful
0812 19:03:51.764418 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":131,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.2@KRBTEST.COM" created.
I20260812 19:03:51.766860 29461 heartbeater.cc:347] Connected to a master server at 127.25.130.62:44945
I20260812 19:03:51.767130 29461 heartbeater.cc:464] Registering TS with master...
I20260812 19:03:51.767787 29461 heartbeater.cc:511] Master 127.25.130.62:44945 requested a full tablet report, sending...
I20260812 19:03:51.769526 29225 ts_manager.cc:194] Registered new tserver with Master: d1d30b056f94452eb18e93204778220b (127.25.130.1:36105)
I20260812 19:03:51.770727 29225 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.1@KRBTEST.COM'} at 127.25.130.1:58065
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
I20260812 19:03:51.784467 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:0
--local_ip_for_outbound_sockets=127.25.130.2
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:44945
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:46411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:51.900452 29467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:51.900758 29467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:51.900835 29467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:51.905491 29467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:51.905576 29467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:51.905651 29467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:03:51.911443 29467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:46411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:44945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:51.912638 29467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:51.913763 29467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:51.924314 29476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:51.924327 29473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:51.924281 29474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:51.924629 29467 server_base.cc:1034] running on GCE node
I20260812 19:03:51.925125 29467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:51.925863 29467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:51.927045 29467 hybrid_clock.cc:648] HybridClock initialized: now 1786561431927024 us; error 35 us; skew 500 ppm
Aug 12 19:03:51 dist-test-slave-49h3 krb5kdc[29165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561431, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:51.930019 29467 init.cc:377] Logged in from keytab as kudu/127.25.130.2@KRBTEST.COM (short username kudu)
I20260812 19:03:51.931025 29467 webserver.cc:460] Webserver started at http://127.25.130.2:43661/ using document root <none> and password file <none>
I20260812 19:03:51.931610 29467 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:51.931679 29467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:51.931898 29467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:51.934204 29467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/data/instance:
uuid: "ce7abb45278847309b3efb3dd4162b06"
format_stamp: "Formatted at 2026-08-12 19:03:51 on dist-test-slave-49h3"
server_key: "abd7c23114c2d0bbb6254245c98198c7"
server_key_iv: "0f098cf2661816ea616be517b17ef619"
server_key_version: "encryptionkey@0"
I20260812 19:03:51.934863 29467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance:
uuid: "ce7abb45278847309b3efb3dd4162b06"
format_stamp: "Formatted at 2026-08-12 19:03:51 on dist-test-slave-49h3"
server_key: "abd7c23114c2d0bbb6254245c98198c7"
server_key_iv: "0f098cf2661816ea616be517b17ef619"
server_key_version: "encryptionkey@0"
I20260812 19:03:51.940645 29467 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 19:03:51.944106 29485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:51.945618 29467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:03:51.945747 29467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "ce7abb45278847309b3efb3dd4162b06"
format_stamp: "Formatted at 2026-08-12 19:03:51 on dist-test-slave-49h3"
server_key: "abd7c23114c2d0bbb6254245c98198c7"
server_key_iv: "0f098cf2661816ea616be517b17ef619"
server_key_version: "encryptionkey@0"
I20260812 19:03:51.945863 29467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:51.957584 29467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:51.960865 29467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:51.961045 29467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:51.961788 29467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:03:51.962693 29467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:03:51.962742 29467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:51.962782 29467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:03:51.962809 29467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:51.971863 29467 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:45999
I20260812 19:03:51.971899 29629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:45999 every 8 connection(s)
I20260812 19:03:51.972851 29467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:03:51.982436 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 29467
I20260812 19:03:51.982569 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance
I20260812 19:03:51.982889 26120 external_mini_cluster.cc:1428] Setting key 81fde81b3ee8fa919c0f686fe3abb2ed
Aug 12 19:03:51 dist-test-slave-49h3 krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561431, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:51.987335 29270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:51.974798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:55849 (local address 127.25.130.62:44945)
0812 19:03:51.974946 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:51.974950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:51.975853 (+ 903us) server_negotiation.cc:407] Connection header received
0812 19:03:51.977242 (+ 1389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:51.977246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:51.977309 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:51.977385 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:51.978734 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:51.979274 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:51.980487 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:51.980720 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:51.980810 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:51.983613 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:51.983616 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:51.983619 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:51.983647 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:51.985084 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:51.985675 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:51.985679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:51.985681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:51.985767 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:51.986151 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:51.986154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:51.986156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:51.986354 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:51.986439 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:51.987021 (+ 582us) server_negotiation.cc:299] Negotiation successful
0812 19:03:51.987164 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 19:03:51.988211 29635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:51.975180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:44945 (local address 127.25.130.2:55849)
0812 19:03:51.975679 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:51.975730 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:51.976976 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:51.977643 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:51.977651 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:51.978036 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:51.978564 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:51.978579 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:51.979417 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:51.979420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:51.980346 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:51.980357 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:51.980896 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:51.980901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:51.981081 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:51.981693 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:51.981711 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:51.983420 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:51.985210 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:51.985217 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:51.985235 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:51.985575 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:51.985861 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:51.985864 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:51.985866 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:51.986041 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:51.986467 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:51.986474 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:51.986756 (+ 282us) client_negotiation.cc:769] Sending connection context
0812 19:03:51.986931 (+ 175us) client_negotiation.cc:240] Negotiation successful
0812 19:03:51.987301 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":116,"threads_started":1}
I20260812 19:03:51.989719 29633 heartbeater.cc:347] Connected to a master server at 127.25.130.62:44945
I20260812 19:03:51.989971 29633 heartbeater.cc:464] Registering TS with master...
I20260812 19:03:51.990495 29633 heartbeater.cc:511] Master 127.25.130.62:44945 requested a full tablet report, sending...
I20260812 19:03:51.991833 29225 ts_manager.cc:194] Registered new tserver with Master: ce7abb45278847309b3efb3dd4162b06 (127.25.130.2:45999)
I20260812 19:03:51.992503 29225 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.2@KRBTEST.COM'} at 127.25.130.2:55849
WARNING: no policy specified for kudu/127.25.130.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
I20260812 19:03:52.013142 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:0
--local_ip_for_outbound_sockets=127.25.130.3
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:44945
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:46411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:52.129154 29639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:52.129503 29639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:52.129571 29639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:52.133829 29639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:52.133906 29639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:52.133975 29639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:03:52.139623 29639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:46411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:44945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:52.140859 29639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:52.142072 29639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:52.152485 29648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:52.152673 29639 server_base.cc:1034] running on GCE node
W20260812 19:03:52.152436 29644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:52.152432 29645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:52.153235 29639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:52.153942 29639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:52.155100 29639 hybrid_clock.cc:648] HybridClock initialized: now 1786561432155075 us; error 42 us; skew 500 ppm
Aug 12 19:03:52 dist-test-slave-49h3 krb5kdc[29165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561432, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:52.157933 29639 init.cc:377] Logged in from keytab as kudu/127.25.130.3@KRBTEST.COM (short username kudu)
I20260812 19:03:52.158862 29639 webserver.cc:460] Webserver started at http://127.25.130.3:46399/ using document root <none> and password file <none>
I20260812 19:03:52.159454 29639 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:52.159518 29639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:52.159732 29639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:52.161929 29639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/data/instance:
uuid: "d53feba92d0d4245bfd1cade62905931"
format_stamp: "Formatted at 2026-08-12 19:03:52 on dist-test-slave-49h3"
server_key: "6a68612d7a1ead8ab0c335c967e6b491"
server_key_iv: "17581878942e39eba947ff520e52f8dd"
server_key_version: "encryptionkey@0"
I20260812 19:03:52.162482 29639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance:
uuid: "d53feba92d0d4245bfd1cade62905931"
format_stamp: "Formatted at 2026-08-12 19:03:52 on dist-test-slave-49h3"
server_key: "6a68612d7a1ead8ab0c335c967e6b491"
server_key_iv: "17581878942e39eba947ff520e52f8dd"
server_key_version: "encryptionkey@0"
I20260812 19:03:52.166409 29639 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 19:03:52.168995 29657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:52.170403 29639 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:03:52.170533 29639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "d53feba92d0d4245bfd1cade62905931"
format_stamp: "Formatted at 2026-08-12 19:03:52 on dist-test-slave-49h3"
server_key: "6a68612d7a1ead8ab0c335c967e6b491"
server_key_iv: "17581878942e39eba947ff520e52f8dd"
server_key_version: "encryptionkey@0"
I20260812 19:03:52.170655 29639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:52.183087 29639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:52.187906 29639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:52.188151 29639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:52.188964 29639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:03:52.190148 29639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:03:52.190199 29639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:52.190248 29639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:03:52.190277 29639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:52.200667 29639 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:39989
I20260812 19:03:52.200693 29799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:39989 every 8 connection(s)
I20260812 19:03:52.201822 29639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:03:52.211117 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 29639
I20260812 19:03:52.211280 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance
I20260812 19:03:52.211714 26120 external_mini_cluster.cc:1428] Setting key 40424b07503487a09ae91fe34dcc9ebb
Aug 12 19:03:52 dist-test-slave-49h3 krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561432, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:52.218477 29270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:52.204132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:35535 (local address 127.25.130.62:44945)
0812 19:03:52.204315 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:52.204321 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:52.205411 (+ 1090us) server_negotiation.cc:407] Connection header received
0812 19:03:52.206971 (+ 1560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:52.206974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:52.207042 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:52.207124 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:52.208564 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:52.209158 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:52.210478 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:52.210727 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:52.210841 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:52.214227 (+ 3386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:52.214230 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:52.214233 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:52.214269 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:52.215877 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:52.216518 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:52.216522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:52.216524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:52.216628 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:52.217082 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:52.217084 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:52.217085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:52.217320 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:52.217400 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:52.218178 (+ 778us) server_negotiation.cc:299] Negotiation successful
0812 19:03:52.218316 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 19:03:52.219668 29803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:52.204614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:44945 (local address 127.25.130.3:35535)
0812 19:03:52.205183 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:52.205237 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:52.206720 (+ 1483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:52.207345 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:52.207353 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:52.207793 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:52.208415 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:52.208435 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:52.209287 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:52.209291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:52.210370 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:52.210381 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:52.210906 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:52.210911 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:52.211144 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:52.212048 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:52.212073 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:52.214015 (+ 1942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:52.216019 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:52.216027 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:52.216048 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:52.216362 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:52.216776 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:52.216779 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:52.216781 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:52.216976 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:52.217429 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:52.217436 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:52.217888 (+ 452us) client_negotiation.cc:769] Sending connection context
0812 19:03:52.218106 (+ 218us) client_negotiation.cc:240] Negotiation successful
0812 19:03:52.218496 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":131,"threads_started":1}
I20260812 19:03:52.221738 29800 heartbeater.cc:347] Connected to a master server at 127.25.130.62:44945
I20260812 19:03:52.222124 29800 heartbeater.cc:464] Registering TS with master...
I20260812 19:03:52.222910 29800 heartbeater.cc:511] Master 127.25.130.62:44945 requested a full tablet report, sending...
I20260812 19:03:52.224457 29225 ts_manager.cc:194] Registered new tserver with Master: d53feba92d0d4245bfd1cade62905931 (127.25.130.3:39989)
I20260812 19:03:52.225319 29225 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.3@KRBTEST.COM'} at 127.25.130.3:35535
I20260812 19:03:52.227263 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 19:03:52.247169 29270 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:03:52.245107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39904 (local address 127.25.130.62:44945)
0812 19:03:52.245334 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:52.245339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:52.245535 (+ 196us) server_negotiation.cc:407] Connection header received
0812 19:03:52.245739 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:52.245742 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:52.245800 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:52.245876 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:52.246970 (+ 1094us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39904: BlockingRecv error: recv got EOF from 127.0.0.1:39904 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260812 19:03:52.249037 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 29298
I20260812 19:03:52.261329 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 29467
I20260812 19:03:52.272207 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 29639
I20260812 19:03:52.286502 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 29180
2026-08-12T19:03:52Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2287 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:03:52 dist-test-slave-49h3 krb5kdc[29816](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:03:52 dist-test-slave-49h3 krb5kdc[29816](info): set up 2 sockets
Aug 12 19:03:52 dist-test-slave-49h3 krb5kdc[29816](info): commencing operation
krb5kdc: starting...
W20260812 19:03:53.356812 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:03:53 dist-test-slave-49h3 krb5kdc[29816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561433, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:03:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:03:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:03:53.493528 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:40991
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:38343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:40991
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:53.611939 29831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:53.612260 29831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:53.612303 29831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:53.616662 29831 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:53.616755 29831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:53.616773 29831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:53.616789 29831 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:53.616803 29831 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:53.622538 29831 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:38343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:40991
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:40991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:53.623718 29831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:53.624792 29831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:53.634140 29837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:53.634114 29841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:53.634141 29838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:53.634351 29831 server_base.cc:1034] running on GCE node
I20260812 19:03:53.634995 29831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:53.636272 29831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:53.637480 29831 hybrid_clock.cc:648] HybridClock initialized: now 1786561433637426 us; error 46 us; skew 500 ppm
Aug 12 19:03:53 dist-test-slave-49h3 krb5kdc[29816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561433, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:53.640631 29831 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:03:53.641780 29831 webserver.cc:460] Webserver started at http://127.25.130.62:42911/ using document root <none> and password file <none>
I20260812 19:03:53.642365 29831 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:53.642427 29831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:53.642619 29831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:53.644858 29831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "daf9915701074c9e96cc6becbafd55b2"
format_stamp: "Formatted at 2026-08-12 19:03:53 on dist-test-slave-49h3"
server_key: "6d1011a3b6064d56bcfb87f245f5978f"
server_key_iv: "6bd5aeaa454287c25297f3df6c8cf37d"
server_key_version: "encryptionkey@0"
I20260812 19:03:53.645473 29831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "daf9915701074c9e96cc6becbafd55b2"
format_stamp: "Formatted at 2026-08-12 19:03:53 on dist-test-slave-49h3"
server_key: "6d1011a3b6064d56bcfb87f245f5978f"
server_key_iv: "6bd5aeaa454287c25297f3df6c8cf37d"
server_key_version: "encryptionkey@0"
I20260812 19:03:53.649735 29831 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:03:53.652526 29849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:53.654036 29831 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:03:53.654218 29831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "daf9915701074c9e96cc6becbafd55b2"
format_stamp: "Formatted at 2026-08-12 19:03:53 on dist-test-slave-49h3"
server_key: "6d1011a3b6064d56bcfb87f245f5978f"
server_key_iv: "6bd5aeaa454287c25297f3df6c8cf37d"
server_key_version: "encryptionkey@0"
I20260812 19:03:53.654343 29831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:53.675690 29831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:53.679435 29831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:53.679658 29831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:53.687567 29831 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:40991
I20260812 19:03:53.687848 29915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:40991 every 8 connection(s)
I20260812 19:03:53.688715 29831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:53.691835 29916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:53.691888 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 29831
I20260812 19:03:53.692001 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:03:53.692305 26120 external_mini_cluster.cc:1428] Setting key 473a3b899c2c677c96d1add86fdfbda5
I20260812 19:03:53.699877 29916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daf9915701074c9e96cc6becbafd55b2: Bootstrap starting.
I20260812 19:03:53.703013 29916 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P daf9915701074c9e96cc6becbafd55b2: Neither blocks nor log segments found. Creating new log.
Aug 12 19:03:53 dist-test-slave-49h3 krb5kdc[29816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561433, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:53.703926 29916 log.cc:824] T 00000000000000000000000000000000 P daf9915701074c9e96cc6becbafd55b2: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:53.706868 29916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daf9915701074c9e96cc6becbafd55b2: No bootstrap required, opened a new log
I20260812 19:03:53.709688 29920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:53.693930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48440 (local address 127.25.130.62:40991)
0812 19:03:53.694452 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:53.694473 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:53.694519 (+ 46us) server_negotiation.cc:407] Connection header received
0812 19:03:53.695334 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:53.695363 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:53.695860 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:53.696300 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:53.697139 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:53.698699 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:53.699967 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:53.700717 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:53.700891 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:53.703744 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:53.703752 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:53.703772 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:53.703816 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:53.706057 (+ 2241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:53.706591 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:53.706599 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:53.706606 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:53.706765 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:53.707033 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:53.707036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:53.707038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:53.707637 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:53.707854 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:53.708157 (+ 303us) server_negotiation.cc:299] Negotiation successful
0812 19:03:53.708494 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":139,"threads_started":1}
I20260812 19:03:53.711710 29916 raft_consensus.cc:348] T 00000000000000000000000000000000 P daf9915701074c9e96cc6becbafd55b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daf9915701074c9e96cc6becbafd55b2" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40991 } }
I20260812 19:03:53.712044 29916 raft_consensus.cc:374] T 00000000000000000000000000000000 P daf9915701074c9e96cc6becbafd55b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:53.712109 29916 raft_consensus.cc:729] T 00000000000000000000000000000000 P daf9915701074c9e96cc6becbafd55b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: daf9915701074c9e96cc6becbafd55b2, State: Initialized, Role: FOLLOWER
I20260812 19:03:53.712615 29916 consensus_queue.cc:260] T 00000000000000000000000000000000 P daf9915701074c9e96cc6becbafd55b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daf9915701074c9e96cc6becbafd55b2" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40991 } }
I20260812 19:03:53.712767 29916 raft_consensus.cc:388] T 00000000000000000000000000000000 P daf9915701074c9e96cc6becbafd55b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:03:53.712838 29916 raft_consensus.cc:482] T 00000000000000000000000000000000 P daf9915701074c9e96cc6becbafd55b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:03:53.712941 29916 raft_consensus.cc:3037] T 00000000000000000000000000000000 P daf9915701074c9e96cc6becbafd55b2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:53.713069 26120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:03:53.714067 29916 raft_consensus.cc:504] T 00000000000000000000000000000000 P daf9915701074c9e96cc6becbafd55b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daf9915701074c9e96cc6becbafd55b2" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40991 } }
I20260812 19:03:53.714421 29916 leader_election.cc:302] T 00000000000000000000000000000000 P daf9915701074c9e96cc6becbafd55b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: daf9915701074c9e96cc6becbafd55b2; no voters:
I20260812 19:03:53.714715 29916 leader_election.cc:288] T 00000000000000000000000000000000 P daf9915701074c9e96cc6becbafd55b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:03:53.714905 29922 raft_consensus.cc:2781] T 00000000000000000000000000000000 P daf9915701074c9e96cc6becbafd55b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:53.715185 29922 raft_consensus.cc:686] T 00000000000000000000000000000000 P daf9915701074c9e96cc6becbafd55b2 [term 1 LEADER]: Becoming Leader. State: Replica: daf9915701074c9e96cc6becbafd55b2, State: Running, Role: LEADER
I20260812 19:03:53.715544 29922 consensus_queue.cc:237] T 00000000000000000000000000000000 P daf9915701074c9e96cc6becbafd55b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daf9915701074c9e96cc6becbafd55b2" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40991 } }
I20260812 19:03:53.716264 29916 sys_catalog.cc:564] T 00000000000000000000000000000000 P daf9915701074c9e96cc6becbafd55b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:53.717317 29925 sys_catalog.cc:455] T 00000000000000000000000000000000 P daf9915701074c9e96cc6becbafd55b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader daf9915701074c9e96cc6becbafd55b2. Latest consensus state: current_term: 1 leader_uuid: "daf9915701074c9e96cc6becbafd55b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daf9915701074c9e96cc6becbafd55b2" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40991 } } }
I20260812 19:03:53.717505 29925 sys_catalog.cc:458] T 00000000000000000000000000000000 P daf9915701074c9e96cc6becbafd55b2 [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:53.717808 29923 sys_catalog.cc:455] T 00000000000000000000000000000000 P daf9915701074c9e96cc6becbafd55b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "daf9915701074c9e96cc6becbafd55b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daf9915701074c9e96cc6becbafd55b2" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40991 } } }
I20260812 19:03:53.717908 29923 sys_catalog.cc:458] T 00000000000000000000000000000000 P daf9915701074c9e96cc6becbafd55b2 [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:53.717981 29934 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:53.721508 29934 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:03:53.728770 29934 catalog_manager.cc:1347] Generated new cluster ID: 5fe173809a11473fbba34627b7e52d57
I20260812 19:03:53.728864 29934 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:53.736596 29920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:53.730666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48450 (local address 127.25.130.62:40991)
0812 19:03:53.730778 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:53.730782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:53.731075 (+ 293us) server_negotiation.cc:407] Connection header received
0812 19:03:53.731180 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:53.731184 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:53.731241 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:53.731330 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:53.731929 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:53.732763 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:53.733958 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:53.734354 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:53.734464 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:53.735447 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:53.735451 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:53.735454 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:03:53.736105 (+ 651us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:53.736147 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:03:53.736167 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:53.736259 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:53.736356 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 19:03:53.736430 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 19:03:53.739555 29934 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:53.740582 29934 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:53.748692 29934 catalog_manager.cc:5868] T 00000000000000000000000000000000 P daf9915701074c9e96cc6becbafd55b2: Generated new TSK 0
I20260812 19:03:53.749761 29934 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:03:53.759375 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 29831
2026-08-12T19:03:53Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1468 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:03:53 dist-test-slave-49h3 krb5kdc[29960](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:03:53 dist-test-slave-49h3 krb5kdc[29960](info): set up 2 sockets
Aug 12 19:03:53 dist-test-slave-49h3 krb5kdc[29960](info): commencing operation
krb5kdc: starting...
W20260812 19:03:54.822867 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:03:54 dist-test-slave-49h3 krb5kdc[29960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:03:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:03:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:03:54.952594 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:42761
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:34577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:42761
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:55.066888 29976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:55.067189 29976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:55.067251 29976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:55.071419 29976 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:55.071498 29976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:55.071527 29976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:55.071549 29976 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:55.071570 29976 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:55.077316 29976 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:34577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:42761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:42761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:55.078604 29976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:55.079734 29976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:03:55.089006 29976 server_base.cc:1034] running on GCE node
W20260812 19:03:55.088958 29986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:55.088953 29982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:55.088953 29984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:55.089910 29976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:55.091075 29976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:55.092258 29976 hybrid_clock.cc:648] HybridClock initialized: now 1786561435092239 us; error 34 us; skew 500 ppm
Aug 12 19:03:55 dist-test-slave-49h3 krb5kdc[29960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561435, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:55.095022 29976 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:03:55.096056 29976 webserver.cc:460] Webserver started at http://127.25.130.62:38501/ using document root <none> and password file <none>
I20260812 19:03:55.096628 29976 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:55.096700 29976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:55.096901 29976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:55.099098 29976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "16735316d500490a9a86f37f83320ac7"
format_stamp: "Formatted at 2026-08-12 19:03:55 on dist-test-slave-49h3"
server_key: "184e53af6e12988f8a6e36db5d9ebaa8"
server_key_iv: "d77cc6ec6f6eb90ce50fb85744e5b787"
server_key_version: "encryptionkey@0"
I20260812 19:03:55.099714 29976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "16735316d500490a9a86f37f83320ac7"
format_stamp: "Formatted at 2026-08-12 19:03:55 on dist-test-slave-49h3"
server_key: "184e53af6e12988f8a6e36db5d9ebaa8"
server_key_iv: "d77cc6ec6f6eb90ce50fb85744e5b787"
server_key_version: "encryptionkey@0"
I20260812 19:03:55.103783 29976 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:03:55.106632 29993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:55.107977 29976 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:03:55.108125 29976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "16735316d500490a9a86f37f83320ac7"
format_stamp: "Formatted at 2026-08-12 19:03:55 on dist-test-slave-49h3"
server_key: "184e53af6e12988f8a6e36db5d9ebaa8"
server_key_iv: "d77cc6ec6f6eb90ce50fb85744e5b787"
server_key_version: "encryptionkey@0"
I20260812 19:03:55.108256 29976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:55.123404 29976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:55.126698 29976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:55.126907 29976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:55.134511 29976 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:42761
I20260812 19:03:55.134791 30054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:42761 every 8 connection(s)
I20260812 19:03:55.135600 29976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:55.138553 30055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:55.140410 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 29976
I20260812 19:03:55.140530 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:03:55.140796 26120 external_mini_cluster.cc:1428] Setting key 326479854438b2a5a0441cf177b49082
I20260812 19:03:55.145877 30055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16735316d500490a9a86f37f83320ac7: Bootstrap starting.
I20260812 19:03:55.148715 30055 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16735316d500490a9a86f37f83320ac7: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:55.149665 30055 log.cc:824] T 00000000000000000000000000000000 P 16735316d500490a9a86f37f83320ac7: Log is configured to *not* fsync() on all Append() calls
Aug 12 19:03:55 dist-test-slave-49h3 krb5kdc[29960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:55.152225 30055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16735316d500490a9a86f37f83320ac7: No bootstrap required, opened a new log
I20260812 19:03:55.157198 30061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:55.142412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55242 (local address 127.25.130.62:42761)
0812 19:03:55.142904 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:55.142923 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:55.142970 (+ 47us) server_negotiation.cc:407] Connection header received
0812 19:03:55.143776 (+ 806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:55.143804 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:55.144261 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:55.144672 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:55.145415 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:55.146926 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:55.147864 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:55.148557 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:55.148739 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:55.151698 (+ 2959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:55.151705 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:55.151722 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:55.151758 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:55.153770 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:55.154244 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:55.154251 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:55.154258 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:55.154418 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:55.154670 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:55.154673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:55.154675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:55.155279 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:55.155487 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:55.155746 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 19:03:55.156094 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":133,"threads_started":1}
I20260812 19:03:55.157107 30055 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16735316d500490a9a86f37f83320ac7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16735316d500490a9a86f37f83320ac7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42761 } }
I20260812 19:03:55.157401 30055 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16735316d500490a9a86f37f83320ac7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:55.157480 30055 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16735316d500490a9a86f37f83320ac7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16735316d500490a9a86f37f83320ac7, State: Initialized, Role: FOLLOWER
I20260812 19:03:55.157953 30055 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16735316d500490a9a86f37f83320ac7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16735316d500490a9a86f37f83320ac7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42761 } }
I20260812 19:03:55.158088 30055 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16735316d500490a9a86f37f83320ac7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:03:55.158136 30055 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16735316d500490a9a86f37f83320ac7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:03:55.158205 30055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16735316d500490a9a86f37f83320ac7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:55.159453 30055 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16735316d500490a9a86f37f83320ac7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16735316d500490a9a86f37f83320ac7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42761 } }
I20260812 19:03:55.159823 30055 leader_election.cc:302] T 00000000000000000000000000000000 P 16735316d500490a9a86f37f83320ac7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16735316d500490a9a86f37f83320ac7; no voters:
I20260812 19:03:55.160158 30055 leader_election.cc:288] T 00000000000000000000000000000000 P 16735316d500490a9a86f37f83320ac7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:03:55.160290 30063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16735316d500490a9a86f37f83320ac7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:55.160506 30063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16735316d500490a9a86f37f83320ac7 [term 1 LEADER]: Becoming Leader. State: Replica: 16735316d500490a9a86f37f83320ac7, State: Running, Role: LEADER
I20260812 19:03:55.160804 30063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16735316d500490a9a86f37f83320ac7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16735316d500490a9a86f37f83320ac7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42761 } }
I20260812 19:03:55.161422 30055 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16735316d500490a9a86f37f83320ac7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:55.162837 30064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16735316d500490a9a86f37f83320ac7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16735316d500490a9a86f37f83320ac7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16735316d500490a9a86f37f83320ac7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42761 } } }
I20260812 19:03:55.162981 30064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16735316d500490a9a86f37f83320ac7 [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:55.163408 30073 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:55.163838 30066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16735316d500490a9a86f37f83320ac7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16735316d500490a9a86f37f83320ac7. Latest consensus state: current_term: 1 leader_uuid: "16735316d500490a9a86f37f83320ac7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16735316d500490a9a86f37f83320ac7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42761 } } }
I20260812 19:03:55.163940 30066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16735316d500490a9a86f37f83320ac7 [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:55.166431 30073 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:03:55.171838 30073 catalog_manager.cc:1347] Generated new cluster ID: 2366d0a80a4d46f2a2331117c2d74ef2
I20260812 19:03:55.171926 30073 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:55.194409 30073 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:55.195767 30073 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:55.204139 30073 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16735316d500490a9a86f37f83320ac7: Generated new TSK 0
I20260812 19:03:55.205004 30073 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
I20260812 19:03:55.245929 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:0
--local_ip_for_outbound_sockets=127.25.130.1
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:42761
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:34577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:55.362144 30087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:55.362501 30087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:55.362602 30087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:55.366976 30087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:55.367060 30087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:55.367127 30087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:03:55.372861 30087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:34577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:42761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:55.374114 30087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:55.375198 30087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:55.385146 30095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:55.385160 30098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:55.385149 30093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:55.385394 30087 server_base.cc:1034] running on GCE node
I20260812 19:03:55.386179 30087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:55.386883 30087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:55.388082 30087 hybrid_clock.cc:648] HybridClock initialized: now 1786561435388062 us; error 31 us; skew 500 ppm
Aug 12 19:03:55 dist-test-slave-49h3 krb5kdc[29960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561435, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:55.390982 30087 init.cc:377] Logged in from keytab as kudu/127.25.130.1@KRBTEST.COM (short username kudu)
I20260812 19:03:55.391924 30087 webserver.cc:460] Webserver started at http://127.25.130.1:36847/ using document root <none> and password file <none>
I20260812 19:03:55.392518 30087 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:55.392593 30087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:55.392800 30087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:55.395061 30087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/data/instance:
uuid: "f7e679561c364aeba148c05890fcda63"
format_stamp: "Formatted at 2026-08-12 19:03:55 on dist-test-slave-49h3"
server_key: "3ae0daa5e56756dea8c519d87108e4aa"
server_key_iv: "ecc1f279427819e7bcf589e8427f346f"
server_key_version: "encryptionkey@0"
I20260812 19:03:55.395659 30087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance:
uuid: "f7e679561c364aeba148c05890fcda63"
format_stamp: "Formatted at 2026-08-12 19:03:55 on dist-test-slave-49h3"
server_key: "3ae0daa5e56756dea8c519d87108e4aa"
server_key_iv: "ecc1f279427819e7bcf589e8427f346f"
server_key_version: "encryptionkey@0"
I20260812 19:03:55.399823 30087 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:03:55.402595 30105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:55.403955 30087 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:03:55.404109 30087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "f7e679561c364aeba148c05890fcda63"
format_stamp: "Formatted at 2026-08-12 19:03:55 on dist-test-slave-49h3"
server_key: "3ae0daa5e56756dea8c519d87108e4aa"
server_key_iv: "ecc1f279427819e7bcf589e8427f346f"
server_key_version: "encryptionkey@0"
I20260812 19:03:55.404225 30087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:55.434021 30087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:55.437328 30087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:55.437553 30087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:55.438208 30087 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:03:55.439095 30087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:03:55.439144 30087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:55.439198 30087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:03:55.439227 30087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:55.448194 30087 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:46453
I20260812 19:03:55.448231 30256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:46453 every 8 connection(s)
I20260812 19:03:55.449150 30087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:03:55.454159 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 30087
I20260812 19:03:55.454288 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance
I20260812 19:03:55.454558 26120 external_mini_cluster.cc:1428] Setting key 10caf08fcf4d7cf482ef33f25b22ce80
Aug 12 19:03:55 dist-test-slave-49h3 krb5kdc[29960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561435, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:55.464346 30061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:55.451131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:48497 (local address 127.25.130.62:42761)
0812 19:03:55.451286 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:55.451290 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:55.452187 (+ 897us) server_negotiation.cc:407] Connection header received
0812 19:03:55.453722 (+ 1535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:55.453726 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:55.453790 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:55.453863 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:55.455284 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:55.456040 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:55.457219 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:55.457487 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:55.457585 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:55.460238 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:55.460241 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:55.460243 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:55.460271 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:55.461828 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:55.462449 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:55.462452 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:55.462453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:55.462545 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:55.463001 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:55.463003 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:55.463004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:55.463223 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:55.463301 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:55.464074 (+ 773us) server_negotiation.cc:299] Negotiation successful
0812 19:03:55.464211 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
WARNING: no policy specified for kudu/127.25.130.2@KRBTEST.COM; defaulting to no policy
I20260812 19:03:55.465545 30259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:55.451481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:42761 (local address 127.25.130.1:48497)
0812 19:03:55.452001 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:55.452058 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:55.453470 (+ 1412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:55.454076 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:55.454083 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:55.454506 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:55.455124 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:55.455147 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:55.456162 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:55.456165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:55.457085 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:55.457096 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:55.457648 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:55.457653 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:55.457836 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:55.458427 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:55.458446 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:55.460044 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:55.461970 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:55.461977 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:55.461996 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:55.462329 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:55.462675 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:55.462679 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:55.462681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:55.462898 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:55.463335 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:55.463343 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:55.463734 (+ 391us) client_negotiation.cc:769] Sending connection context
0812 19:03:55.463978 (+ 244us) client_negotiation.cc:240] Negotiation successful
0812 19:03:55.464399 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":125,"threads_started":1}
I20260812 19:03:55.467051 30257 heartbeater.cc:347] Connected to a master server at 127.25.130.62:42761
I20260812 19:03:55.467357 30257 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.2@KRBTEST.COM" created.
I20260812 19:03:55.467942 30257 heartbeater.cc:511] Master 127.25.130.62:42761 requested a full tablet report, sending...
I20260812 19:03:55.469897 30011 ts_manager.cc:194] Registered new tserver with Master: f7e679561c364aeba148c05890fcda63 (127.25.130.1:46453)
I20260812 19:03:55.471134 30011 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.1@KRBTEST.COM'} at 127.25.130.1:48497
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
I20260812 19:03:55.485150 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:0
--local_ip_for_outbound_sockets=127.25.130.2
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:42761
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:34577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:55.601172 30262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:55.601500 30262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:55.601576 30262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:55.605688 30262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:55.605762 30262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:55.605845 30262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:03:55.611505 30262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:34577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:42761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:55.612701 30262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:55.613818 30262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:03:55.623932 30262 server_base.cc:1034] running on GCE node
W20260812 19:03:55.623950 30268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:55.623950 30271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:55.624119 30269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:55.624603 30262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:55.625322 30262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:55.626505 30262 hybrid_clock.cc:648] HybridClock initialized: now 1786561435626484 us; error 35 us; skew 500 ppm
Aug 12 19:03:55 dist-test-slave-49h3 krb5kdc[29960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561435, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:55.629491 30262 init.cc:377] Logged in from keytab as kudu/127.25.130.2@KRBTEST.COM (short username kudu)
I20260812 19:03:55.630496 30262 webserver.cc:460] Webserver started at http://127.25.130.2:46761/ using document root <none> and password file <none>
I20260812 19:03:55.631047 30262 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:55.631106 30262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:55.631283 30262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:55.633411 30262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/data/instance:
uuid: "d6e0c7ebd8774f6abe68008c5a5546c1"
format_stamp: "Formatted at 2026-08-12 19:03:55 on dist-test-slave-49h3"
server_key: "c987317fb6882652c544fff8a04f0025"
server_key_iv: "1ac7fc3cc21e28b939999838531efa57"
server_key_version: "encryptionkey@0"
I20260812 19:03:55.634043 30262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance:
uuid: "d6e0c7ebd8774f6abe68008c5a5546c1"
format_stamp: "Formatted at 2026-08-12 19:03:55 on dist-test-slave-49h3"
server_key: "c987317fb6882652c544fff8a04f0025"
server_key_iv: "1ac7fc3cc21e28b939999838531efa57"
server_key_version: "encryptionkey@0"
I20260812 19:03:55.638013 30262 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 19:03:55.640655 30280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:55.641999 30262 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:03:55.642114 30262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "d6e0c7ebd8774f6abe68008c5a5546c1"
format_stamp: "Formatted at 2026-08-12 19:03:55 on dist-test-slave-49h3"
server_key: "c987317fb6882652c544fff8a04f0025"
server_key_iv: "1ac7fc3cc21e28b939999838531efa57"
server_key_version: "encryptionkey@0"
I20260812 19:03:55.642258 30262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:55.657207 30262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:55.661690 30262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:55.661938 30262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:55.662786 30262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:03:55.663949 30262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:03:55.664017 30262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:55.664080 30262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:03:55.664112 30262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:55.676776 30262 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:45705
I20260812 19:03:55.676826 30425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:45705 every 8 connection(s)
I20260812 19:03:55.677915 30262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:03:55.683065 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 30262
I20260812 19:03:55.683214 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance
I20260812 19:03:55.683519 26120 external_mini_cluster.cc:1428] Setting key e3ad1b559ca20c78ef6ed5d28a652a0f
Aug 12 19:03:55 dist-test-slave-49h3 krb5kdc[29960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561435, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.130.3@KRBTEST.COM; defaulting to no policy
I20260812 19:03:55.696995 30061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:55.680040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:55757 (local address 127.25.130.62:42761)
0812 19:03:55.680182 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:55.680186 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:55.681310 (+ 1124us) server_negotiation.cc:407] Connection header received
0812 19:03:55.684332 (+ 3022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:55.684336 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:55.684397 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:55.684484 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:55.686259 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:55.687042 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:55.688863 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:55.689190 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:55.689278 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:55.692653 (+ 3375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:55.692657 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:55.692660 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:55.692696 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:55.694513 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:55.695224 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:55.695228 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:55.695230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:55.695324 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:55.695761 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:55.695765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:55.695766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:55.695991 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:55.696060 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:55.696684 (+ 624us) server_negotiation.cc:299] Negotiation successful
0812 19:03:55.696828 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 19:03:55.698328 30428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:55.680611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:42761 (local address 127.25.130.2:55757)
0812 19:03:55.681133 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:55.681191 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:55.684049 (+ 2858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:55.684707 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:55.684718 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:55.685266 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:55.686077 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:55.686101 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:55.687216 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:55.687220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:55.688698 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:55.688712 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:55.689563 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:55.689567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:55.689744 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:55.690375 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:55.690397 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:55.692406 (+ 2009us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:55.694671 (+ 2265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:55.694679 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:55.694699 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:55.695078 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:55.695434 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:55.695437 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:55.695439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:55.695640 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:55.696121 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:55.696126 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:55.696399 (+ 273us) client_negotiation.cc:769] Sending connection context
0812 19:03:55.696777 (+ 378us) client_negotiation.cc:240] Negotiation successful
0812 19:03:55.697197 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":135,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.3@KRBTEST.COM" created.
I20260812 19:03:55.699774 30426 heartbeater.cc:347] Connected to a master server at 127.25.130.62:42761
I20260812 19:03:55.700050 30426 heartbeater.cc:464] Registering TS with master...
I20260812 19:03:55.700722 30426 heartbeater.cc:511] Master 127.25.130.62:42761 requested a full tablet report, sending...
I20260812 19:03:55.701990 30011 ts_manager.cc:194] Registered new tserver with Master: d6e0c7ebd8774f6abe68008c5a5546c1 (127.25.130.2:45705)
I20260812 19:03:55.702581 30011 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.2@KRBTEST.COM'} at 127.25.130.2:55757
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
I20260812 19:03:55.716704 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:0
--local_ip_for_outbound_sockets=127.25.130.3
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:42761
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:34577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:55.833161 30433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:55.833489 30433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:55.833554 30433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:55.837759 30433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:55.837836 30433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:55.837908 30433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:03:55.843400 30433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:34577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:42761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:55.844519 30433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:55.845638 30433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:55.855489 30442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:55.855871 30433 server_base.cc:1034] running on GCE node
W20260812 19:03:55.857740 30440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:55.857787 30439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:55.858279 30433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:55.858970 30433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:55.860157 30433 hybrid_clock.cc:648] HybridClock initialized: now 1786561435860137 us; error 34 us; skew 500 ppm
Aug 12 19:03:55 dist-test-slave-49h3 krb5kdc[29960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561435, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:55.863178 30433 init.cc:377] Logged in from keytab as kudu/127.25.130.3@KRBTEST.COM (short username kudu)
I20260812 19:03:55.864156 30433 webserver.cc:460] Webserver started at http://127.25.130.3:39073/ using document root <none> and password file <none>
I20260812 19:03:55.864729 30433 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:55.864800 30433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:55.865010 30433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:55.867244 30433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/data/instance:
uuid: "45ce70f6ada4443bb673d7fdd656e135"
format_stamp: "Formatted at 2026-08-12 19:03:55 on dist-test-slave-49h3"
server_key: "21f71dce8dd35ca4e1d0270847356e25"
server_key_iv: "8f0d0a36125b5a71b8422eda1351dc63"
server_key_version: "encryptionkey@0"
I20260812 19:03:55.867810 30433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance:
uuid: "45ce70f6ada4443bb673d7fdd656e135"
format_stamp: "Formatted at 2026-08-12 19:03:55 on dist-test-slave-49h3"
server_key: "21f71dce8dd35ca4e1d0270847356e25"
server_key_iv: "8f0d0a36125b5a71b8422eda1351dc63"
server_key_version: "encryptionkey@0"
I20260812 19:03:55.871703 30433 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260812 19:03:55.874226 30449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:55.875530 30433 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 19:03:55.875649 30433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "45ce70f6ada4443bb673d7fdd656e135"
format_stamp: "Formatted at 2026-08-12 19:03:55 on dist-test-slave-49h3"
server_key: "21f71dce8dd35ca4e1d0270847356e25"
server_key_iv: "8f0d0a36125b5a71b8422eda1351dc63"
server_key_version: "encryptionkey@0"
I20260812 19:03:55.875763 30433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:55.906536 30433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:55.910200 30433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:55.910372 30433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:55.910955 30433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:03:55.911895 30433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:03:55.911945 30433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:55.911998 30433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:03:55.912029 30433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:55.922045 30433 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:42521
I20260812 19:03:55.922096 30598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:42521 every 8 connection(s)
I20260812 19:03:55.923213 30433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:03:55.924883 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 30433
I20260812 19:03:55.924999 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance
I20260812 19:03:55.925328 26120 external_mini_cluster.cc:1428] Setting key 0bdd37e4a7f9768ecbfa0d226d1f440f
Aug 12 19:03:55 dist-test-slave-49h3 krb5kdc[29960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561435, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:55.942206 30061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:55.925257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:35433 (local address 127.25.130.62:42761)
0812 19:03:55.925595 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:55.925601 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:55.926503 (+ 902us) server_negotiation.cc:407] Connection header received
0812 19:03:55.928098 (+ 1595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:55.928100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:55.928150 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:55.928236 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:55.929588 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:55.930143 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:55.931369 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:55.931594 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:55.931677 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:55.934361 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:55.934364 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:55.934366 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:55.934396 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:55.939312 (+ 4916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:55.940236 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:55.940274 (+ 38us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:55.940276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:55.940404 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:55.940856 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:55.940859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:55.940861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:55.941123 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:55.941216 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:55.941886 (+ 670us) server_negotiation.cc:299] Negotiation successful
0812 19:03:55.942027 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230}
I20260812 19:03:55.943087 30601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:55.925709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:42761 (local address 127.25.130.3:35433)
0812 19:03:55.926353 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:55.926415 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:55.927875 (+ 1460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:55.928454 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:55.928462 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:55.928867 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:55.929416 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:55.929432 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:55.930292 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:55.930296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:55.931238 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:55.931249 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:55.931754 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:55.931757 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:55.931923 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:55.932534 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:55.932552 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:55.934170 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:55.939587 (+ 5417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:55.939596 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:55.939620 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:55.940081 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:55.940495 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:55.940499 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:55.940501 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:55.940741 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:55.941243 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:55.941251 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:55.941612 (+ 361us) client_negotiation.cc:769] Sending connection context
0812 19:03:55.941795 (+ 183us) client_negotiation.cc:240] Negotiation successful
0812 19:03:55.942181 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":190,"threads_started":1}
I20260812 19:03:55.944301 30599 heartbeater.cc:347] Connected to a master server at 127.25.130.62:42761
I20260812 19:03:55.944571 30599 heartbeater.cc:464] Registering TS with master...
I20260812 19:03:55.945122 30599 heartbeater.cc:511] Master 127.25.130.62:42761 requested a full tablet report, sending...
I20260812 19:03:55.946345 30011 ts_manager.cc:194] Registered new tserver with Master: 45ce70f6ada4443bb673d7fdd656e135 (127.25.130.3:42521)
I20260812 19:03:55.946985 30011 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.3@KRBTEST.COM'} at 127.25.130.3:35433
I20260812 19:03:55.952338 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 19:03:55 dist-test-slave-49h3 krb5kdc[29960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561435, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 19:03:55 dist-test-slave-49h3 krb5kdc[29960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561435, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:55.980468 30061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:55.971214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53942 (local address 127.25.130.62:42761)
0812 19:03:55.971395 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:55.971400 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:55.971492 (+ 92us) server_negotiation.cc:407] Connection header received
0812 19:03:55.971639 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:55.971642 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:55.971690 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:55.971760 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:55.972539 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:55.973073 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:55.974040 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:55.974275 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:55.974372 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:55.977181 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:55.977185 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:55.977187 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:55.977222 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:55.978570 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:55.979070 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:55.979073 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:55.979075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:55.979166 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:55.979475 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:55.979478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:55.979479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:55.979725 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:55.979858 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:55.980173 (+ 315us) server_negotiation.cc:299] Negotiation successful
0812 19:03:55.980293 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260812 19:03:55.980926 30011 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53942
I20260812 19:03:55.981961 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 30087
I20260812 19:03:55.990787 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 30262
I20260812 19:03:56.001842 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 30433
I20260812 19:03:56.016609 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 29976
2026-08-12T19:03:56Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2259 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:03:56 dist-test-slave-49h3 krb5kdc[30616](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:03:56 dist-test-slave-49h3 krb5kdc[30616](info): set up 2 sockets
Aug 12 19:03:56 dist-test-slave-49h3 krb5kdc[30616](info): commencing operation
krb5kdc: starting...
W20260812 19:03:57.075515 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:03:57 dist-test-slave-49h3 krb5kdc[30616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:03:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:03:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:03:57.208665 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:44009
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:36225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:44009
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:57.330153 30633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:57.330461 30633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:57.330503 30633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:57.334780 30633 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:57.334868 30633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:57.334888 30633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:57.334904 30633 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:57.334918 30633 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:57.340651 30633 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:36225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:44009
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:44009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:57.341884 30633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:57.342937 30633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:57.351788 30639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:57.351890 30633 server_base.cc:1034] running on GCE node
W20260812 19:03:57.351809 30642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:57.351809 30640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:57.352692 30633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:57.353928 30633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:57.355099 30633 hybrid_clock.cc:648] HybridClock initialized: now 1786561437355073 us; error 41 us; skew 500 ppm
Aug 12 19:03:57 dist-test-slave-49h3 krb5kdc[30616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561437, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:57.364233 30633 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:03:57.365298 30633 webserver.cc:460] Webserver started at http://127.25.130.62:41835/ using document root <none> and password file <none>
I20260812 19:03:57.365947 30633 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:57.366050 30633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:57.366288 30633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:57.368487 30633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "e3afb4cb674f4444a238c17d61e57607"
format_stamp: "Formatted at 2026-08-12 19:03:57 on dist-test-slave-49h3"
server_key: "45a3b0bcf59c95ff12ec83becaa3f994"
server_key_iv: "5d387614c9e068830ce48d7b667f672b"
server_key_version: "encryptionkey@0"
I20260812 19:03:57.369063 30633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "e3afb4cb674f4444a238c17d61e57607"
format_stamp: "Formatted at 2026-08-12 19:03:57 on dist-test-slave-49h3"
server_key: "45a3b0bcf59c95ff12ec83becaa3f994"
server_key_iv: "5d387614c9e068830ce48d7b667f672b"
server_key_version: "encryptionkey@0"
I20260812 19:03:57.372900 30633 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 19:03:57.375454 30653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:57.376677 30633 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:03:57.376802 30633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "e3afb4cb674f4444a238c17d61e57607"
format_stamp: "Formatted at 2026-08-12 19:03:57 on dist-test-slave-49h3"
server_key: "45a3b0bcf59c95ff12ec83becaa3f994"
server_key_iv: "5d387614c9e068830ce48d7b667f672b"
server_key_version: "encryptionkey@0"
I20260812 19:03:57.376909 30633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:57.400372 30633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:57.403846 30633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:57.404058 30633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:57.411808 30633 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:44009
I20260812 19:03:57.411878 30718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:44009 every 8 connection(s)
I20260812 19:03:57.412873 30633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:57.415827 30719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:57.416985 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 30633
I20260812 19:03:57.417125 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:03:57.417475 26120 external_mini_cluster.cc:1428] Setting key 6f899a96dfb6bfd538c6a994e089d3be
I20260812 19:03:57.422924 30719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3afb4cb674f4444a238c17d61e57607: Bootstrap starting.
I20260812 19:03:57.425833 30719 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3afb4cb674f4444a238c17d61e57607: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:57.426846 30719 log.cc:824] T 00000000000000000000000000000000 P e3afb4cb674f4444a238c17d61e57607: Log is configured to *not* fsync() on all Append() calls
Aug 12 19:03:57 dist-test-slave-49h3 krb5kdc[30616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:57.429165 30719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3afb4cb674f4444a238c17d61e57607: No bootstrap required, opened a new log
I20260812 19:03:57.432752 30722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:57.419092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46910 (local address 127.25.130.62:44009)
0812 19:03:57.419630 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:57.419671 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:57.419723 (+ 52us) server_negotiation.cc:407] Connection header received
0812 19:03:57.420414 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:57.420433 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:57.420793 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:57.421105 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:57.422110 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:57.423136 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:57.424332 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:57.424855 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:57.424986 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:57.428026 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:57.428031 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:57.428049 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:57.428078 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:57.429810 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:57.430282 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:57.430288 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:57.430293 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:57.430445 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:57.430728 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:57.430731 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:57.430732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:57.431161 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:57.431375 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:57.431667 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 19:03:57.431961 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":158,"threads_started":1}
I20260812 19:03:57.432852 30719 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3afb4cb674f4444a238c17d61e57607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3afb4cb674f4444a238c17d61e57607" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 44009 } }
I20260812 19:03:57.433167 30719 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3afb4cb674f4444a238c17d61e57607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:57.433226 30719 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3afb4cb674f4444a238c17d61e57607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3afb4cb674f4444a238c17d61e57607, State: Initialized, Role: FOLLOWER
I20260812 19:03:57.433725 30719 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3afb4cb674f4444a238c17d61e57607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3afb4cb674f4444a238c17d61e57607" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 44009 } }
I20260812 19:03:57.433895 30719 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3afb4cb674f4444a238c17d61e57607 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:03:57.433965 30719 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3afb4cb674f4444a238c17d61e57607 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:03:57.434069 30719 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3afb4cb674f4444a238c17d61e57607 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:57.435076 30719 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3afb4cb674f4444a238c17d61e57607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3afb4cb674f4444a238c17d61e57607" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 44009 } }
I20260812 19:03:57.435423 30719 leader_election.cc:302] T 00000000000000000000000000000000 P e3afb4cb674f4444a238c17d61e57607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3afb4cb674f4444a238c17d61e57607; no voters:
I20260812 19:03:57.435708 30719 leader_election.cc:288] T 00000000000000000000000000000000 P e3afb4cb674f4444a238c17d61e57607 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:03:57.435819 30725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3afb4cb674f4444a238c17d61e57607 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:57.436007 30725 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3afb4cb674f4444a238c17d61e57607 [term 1 LEADER]: Becoming Leader. State: Replica: e3afb4cb674f4444a238c17d61e57607, State: Running, Role: LEADER
I20260812 19:03:57.436319 30725 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3afb4cb674f4444a238c17d61e57607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3afb4cb674f4444a238c17d61e57607" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 44009 } }
I20260812 19:03:57.437073 30719 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3afb4cb674f4444a238c17d61e57607 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:57.438035 30728 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3afb4cb674f4444a238c17d61e57607 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3afb4cb674f4444a238c17d61e57607. Latest consensus state: current_term: 1 leader_uuid: "e3afb4cb674f4444a238c17d61e57607" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3afb4cb674f4444a238c17d61e57607" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 44009 } } }
I20260812 19:03:57.438182 30728 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3afb4cb674f4444a238c17d61e57607 [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:57.438047 30727 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3afb4cb674f4444a238c17d61e57607 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3afb4cb674f4444a238c17d61e57607" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3afb4cb674f4444a238c17d61e57607" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 44009 } } }
I20260812 19:03:57.438304 30727 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3afb4cb674f4444a238c17d61e57607 [sys.catalog]: This master's current role is: LEADER
W20260812 19:03:57.442049 30748 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e3afb4cb674f4444a238c17d61e57607: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:03:57.442133 30748 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:03:57.442216 30740 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:57.442963 30740 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:03:57.448818 30740 catalog_manager.cc:1347] Generated new cluster ID: 2dc0cd9c5d0e4fd0a6a0e36d24aa9008
I20260812 19:03:57.448907 30740 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:57.456235 30740 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:57.457554 30740 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:57.472074 30740 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3afb4cb674f4444a238c17d61e57607: Generated new TSK 0
I20260812 19:03:57.473053 30740 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
I20260812 19:03:57.504330 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:0
--local_ip_for_outbound_sockets=127.25.130.1
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:44009
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:36225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:57.625728 30754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:57.626051 30754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:57.626111 30754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:57.630311 30754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:57.630393 30754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:57.630455 30754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:03:57.636053 30754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:36225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:44009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:57.637312 30754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:57.638469 30754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:57.648808 30759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:57.649012 30754 server_base.cc:1034] running on GCE node
W20260812 19:03:57.648811 30764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:57.648811 30766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:57.649777 30754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:57.650535 30754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:57.651748 30754 hybrid_clock.cc:648] HybridClock initialized: now 1786561437651726 us; error 38 us; skew 500 ppm
Aug 12 19:03:57 dist-test-slave-49h3 krb5kdc[30616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561437, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:57.654754 30754 init.cc:377] Logged in from keytab as kudu/127.25.130.1@KRBTEST.COM (short username kudu)
I20260812 19:03:57.655772 30754 webserver.cc:460] Webserver started at http://127.25.130.1:40919/ using document root <none> and password file <none>
I20260812 19:03:57.656347 30754 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:57.656420 30754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:57.656634 30754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:57.658902 30754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-0/data/instance:
uuid: "791e6680e13b43898cfaf5ee1387239f"
format_stamp: "Formatted at 2026-08-12 19:03:57 on dist-test-slave-49h3"
server_key: "b2a4db8ed55add39f17c819234eebd95"
server_key_iv: "af4d46f4fc031f1fdb4eedba75138969"
server_key_version: "encryptionkey@0"
I20260812 19:03:57.659512 30754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance:
uuid: "791e6680e13b43898cfaf5ee1387239f"
format_stamp: "Formatted at 2026-08-12 19:03:57 on dist-test-slave-49h3"
server_key: "b2a4db8ed55add39f17c819234eebd95"
server_key_iv: "af4d46f4fc031f1fdb4eedba75138969"
server_key_version: "encryptionkey@0"
I20260812 19:03:57.663520 30754 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:03:57.666164 30773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:57.667462 30754 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:03:57.667595 30754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "791e6680e13b43898cfaf5ee1387239f"
format_stamp: "Formatted at 2026-08-12 19:03:57 on dist-test-slave-49h3"
server_key: "b2a4db8ed55add39f17c819234eebd95"
server_key_iv: "af4d46f4fc031f1fdb4eedba75138969"
server_key_version: "encryptionkey@0"
I20260812 19:03:57.667712 30754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:57.689121 30754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:57.692207 30754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:57.692411 30754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:57.693060 30754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:03:57.693975 30754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:03:57.694038 30754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:57.694106 30754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:03:57.694134 30754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:57.703117 30754 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:35713
I20260812 19:03:57.703138 30915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:35713 every 8 connection(s)
I20260812 19:03:57.704075 30754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:03:57.712450 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 30754
I20260812 19:03:57.712587 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance
I20260812 19:03:57.712929 26120 external_mini_cluster.cc:1428] Setting key 988ef1a4ff70f713db56abb81ec497bf
Aug 12 19:03:57 dist-test-slave-49h3 krb5kdc[30616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561437, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:57.719385 30722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:57.706034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:47555 (local address 127.25.130.62:44009)
0812 19:03:57.706218 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:57.706223 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:57.707151 (+ 928us) server_negotiation.cc:407] Connection header received
0812 19:03:57.708525 (+ 1374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:57.708528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:57.708580 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:57.708659 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:57.709959 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:57.710513 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:57.711638 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:57.711891 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:57.711974 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:57.715121 (+ 3147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:57.715126 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:57.715129 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:57.715159 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:57.716909 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:57.717641 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:57.717644 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:57.717645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:57.717732 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:57.718147 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:57.718148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:57.718149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:57.718339 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:57.718414 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:57.719050 (+ 636us) server_negotiation.cc:299] Negotiation successful
0812 19:03:57.719211 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 19:03:57.720206 30918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:57.706479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:44009 (local address 127.25.130.1:47555)
0812 19:03:57.706973 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:57.707022 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:57.708308 (+ 1286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:57.708822 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:57.708829 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:57.709201 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:57.709780 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:57.709796 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:57.710631 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:57.710633 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:57.711514 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:57.711523 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:57.712025 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:57.712028 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:57.712196 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:57.712898 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:57.712919 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:57.714922 (+ 2003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:57.717063 (+ 2141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:57.717071 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:57.717093 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:57.717540 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:57.717820 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:57.717823 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:57.717825 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:57.718057 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:57.718440 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:57.718446 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:57.718752 (+ 306us) client_negotiation.cc:769] Sending connection context
0812 19:03:57.718925 (+ 173us) client_negotiation.cc:240] Negotiation successful
0812 19:03:57.719242 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":133,"threads_started":1}
I20260812 19:03:57.721555 30916 heartbeater.cc:347] Connected to a master server at 127.25.130.62:44009
I20260812 19:03:57.721810 30916 heartbeater.cc:464] Registering TS with master...
I20260812 19:03:57.722360 30916 heartbeater.cc:511] Master 127.25.130.62:44009 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.25.130.2@KRBTEST.COM; defaulting to no policy
I20260812 19:03:57.723979 30676 ts_manager.cc:194] Registered new tserver with Master: 791e6680e13b43898cfaf5ee1387239f (127.25.130.1:35713)
I20260812 19:03:57.725210 30676 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.1@KRBTEST.COM'} at 127.25.130.1:47555
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
I20260812 19:03:57.742558 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:0
--local_ip_for_outbound_sockets=127.25.130.2
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:44009
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:36225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:57.859086 30922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:57.859381 30922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:57.859431 30922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:57.863742 30922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:57.863827 30922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:57.863888 30922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:03:57.869493 30922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:36225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:44009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:57.870635 30922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:57.871769 30922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:57.881711 30933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:57.881718 30928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:57.881829 30922 server_base.cc:1034] running on GCE node
W20260812 19:03:57.881716 30930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:57.882491 30922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:57.883157 30922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:57.884322 30922 hybrid_clock.cc:648] HybridClock initialized: now 1786561437884303 us; error 31 us; skew 500 ppm
Aug 12 19:03:57 dist-test-slave-49h3 krb5kdc[30616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561437, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:57.887125 30922 init.cc:377] Logged in from keytab as kudu/127.25.130.2@KRBTEST.COM (short username kudu)
I20260812 19:03:57.888109 30922 webserver.cc:460] Webserver started at http://127.25.130.2:34735/ using document root <none> and password file <none>
I20260812 19:03:57.888669 30922 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:57.888751 30922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:57.888969 30922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:57.891163 30922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-1/data/instance:
uuid: "7ff131e19b5946d28a050229cecb3858"
format_stamp: "Formatted at 2026-08-12 19:03:57 on dist-test-slave-49h3"
server_key: "243634fe89f0e4485eb967b6f973e8a8"
server_key_iv: "89f6612318078c2513bcc37ae5a354e0"
server_key_version: "encryptionkey@0"
I20260812 19:03:57.891738 30922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance:
uuid: "7ff131e19b5946d28a050229cecb3858"
format_stamp: "Formatted at 2026-08-12 19:03:57 on dist-test-slave-49h3"
server_key: "243634fe89f0e4485eb967b6f973e8a8"
server_key_iv: "89f6612318078c2513bcc37ae5a354e0"
server_key_version: "encryptionkey@0"
I20260812 19:03:57.895546 30922 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:03:57.898082 30942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:57.899281 30922 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:03:57.899406 30922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "7ff131e19b5946d28a050229cecb3858"
format_stamp: "Formatted at 2026-08-12 19:03:57 on dist-test-slave-49h3"
server_key: "243634fe89f0e4485eb967b6f973e8a8"
server_key_iv: "89f6612318078c2513bcc37ae5a354e0"
server_key_version: "encryptionkey@0"
I20260812 19:03:57.899507 30922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:57.943015 30922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:57.947610 30922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:57.947839 30922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:57.948738 30922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:03:57.949806 30922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:03:57.949859 30922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:57.949900 30922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:03:57.949925 30922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:57.958814 30922 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:34203
I20260812 19:03:57.958837 31101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:34203 every 8 connection(s)
I20260812 19:03:57.959735 30922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:03:57.960760 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 30922
I20260812 19:03:57.960887 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance
I20260812 19:03:57.961225 26120 external_mini_cluster.cc:1428] Setting key 0e1c1ed4a3dace6274934d9cd359c282
Aug 12 19:03:57 dist-test-slave-49h3 krb5kdc[30616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561437, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.130.3@KRBTEST.COM; defaulting to no policy
I20260812 19:03:57.974886 30722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:57.963236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:56273 (local address 127.25.130.62:44009)
0812 19:03:57.963368 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:57.963371 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:57.963385 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:03:57.964800 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:57.964803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:57.964848 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:57.964923 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:57.966314 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:57.966836 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:57.967991 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:57.968357 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:57.968474 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:57.971015 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:57.971019 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:57.971021 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:57.971048 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:57.972739 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:57.973233 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:57.973236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:57.973237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:57.973310 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:57.973706 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:57.973708 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:57.973710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:57.973915 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:57.973997 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:57.974563 (+ 566us) server_negotiation.cc:299] Negotiation successful
0812 19:03:57.974708 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.3@KRBTEST.COM" created.
I20260812 19:03:57.975875 31104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:57.962295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:44009 (local address 127.25.130.2:56273)
0812 19:03:57.962893 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:57.962959 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:57.964556 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:57.965138 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:57.965147 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:57.965610 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:57.966161 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:57.966176 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:57.966938 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:57.966940 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:57.967852 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:57.967862 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:57.968543 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:57.968548 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:57.968722 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:57.969280 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:57.969297 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:57.970824 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:57.972863 (+ 2039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:57.972868 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:57.972882 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:57.973152 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:57.973387 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:57.973390 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:57.973392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:57.973612 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:57.974019 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:57.974026 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:57.974319 (+ 293us) client_negotiation.cc:769] Sending connection context
0812 19:03:57.974488 (+ 169us) client_negotiation.cc:240] Negotiation successful
0812 19:03:57.974861 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":150,"threads_started":1}
I20260812 19:03:57.977101 31102 heartbeater.cc:347] Connected to a master server at 127.25.130.62:44009
I20260812 19:03:57.977367 31102 heartbeater.cc:464] Registering TS with master...
I20260812 19:03:57.978052 31102 heartbeater.cc:511] Master 127.25.130.62:44009 requested a full tablet report, sending...
I20260812 19:03:57.979224 30676 ts_manager.cc:194] Registered new tserver with Master: 7ff131e19b5946d28a050229cecb3858 (127.25.130.2:34203)
I20260812 19:03:57.979806 30676 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.2@KRBTEST.COM'} at 127.25.130.2:56273
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
I20260812 19:03:57.992604 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:0
--local_ip_for_outbound_sockets=127.25.130.3
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:44009
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:36225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:58.108911 31109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:58.109226 31109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:58.109273 31109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:58.113541 31109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:58.113624 31109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:58.113691 31109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:03:58.119297 31109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:36225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:44009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:58.120428 31109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:58.121491 31109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:58.131501 31118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:58.131546 31109 server_base.cc:1034] running on GCE node
W20260812 19:03:58.131487 31116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:58.131487 31115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:58.132342 31109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:58.133069 31109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:58.134269 31109 hybrid_clock.cc:648] HybridClock initialized: now 1786561438134249 us; error 35 us; skew 500 ppm
Aug 12 19:03:58 dist-test-slave-49h3 krb5kdc[30616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561438, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:58.137269 31109 init.cc:377] Logged in from keytab as kudu/127.25.130.3@KRBTEST.COM (short username kudu)
I20260812 19:03:58.138379 31109 webserver.cc:460] Webserver started at http://127.25.130.3:42639/ using document root <none> and password file <none>
I20260812 19:03:58.138991 31109 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:58.139068 31109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:58.139258 31109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:58.141515 31109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-2/data/instance:
uuid: "8582f752cbd74045b4388a1014e00e5d"
format_stamp: "Formatted at 2026-08-12 19:03:58 on dist-test-slave-49h3"
server_key: "d9355f7dbb7f0d143ad2d8d216404988"
server_key_iv: "d8a2d526f59dcc7437b8b9ec1589f12f"
server_key_version: "encryptionkey@0"
I20260812 19:03:58.142122 31109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance:
uuid: "8582f752cbd74045b4388a1014e00e5d"
format_stamp: "Formatted at 2026-08-12 19:03:58 on dist-test-slave-49h3"
server_key: "d9355f7dbb7f0d143ad2d8d216404988"
server_key_iv: "d8a2d526f59dcc7437b8b9ec1589f12f"
server_key_version: "encryptionkey@0"
I20260812 19:03:58.146203 31109 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 19:03:58.148811 31128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:58.150310 31109 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 19:03:58.150441 31109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "8582f752cbd74045b4388a1014e00e5d"
format_stamp: "Formatted at 2026-08-12 19:03:58 on dist-test-slave-49h3"
server_key: "d9355f7dbb7f0d143ad2d8d216404988"
server_key_iv: "d8a2d526f59dcc7437b8b9ec1589f12f"
server_key_version: "encryptionkey@0"
I20260812 19:03:58.150561 31109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:58.164395 31109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:58.167789 31109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:58.167984 31109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:58.168603 31109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:03:58.169525 31109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:03:58.169584 31109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:58.169641 31109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:03:58.169675 31109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:58.178426 31109 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:42349
I20260812 19:03:58.178453 31271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:42349 every 8 connection(s)
I20260812 19:03:58.179342 31109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:03:58.180022 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 31109
I20260812 19:03:58.180155 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance
I20260812 19:03:58.180454 26120 external_mini_cluster.cc:1428] Setting key f31f75579155273e10f8f2f83c6a63a2
Aug 12 19:03:58 dist-test-slave-49h3 krb5kdc[30616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561438, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:58.193675 30722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:58.181568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:39559 (local address 127.25.130.62:44009)
0812 19:03:58.181726 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:58.181730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:58.182667 (+ 937us) server_negotiation.cc:407] Connection header received
0812 19:03:58.184064 (+ 1397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:58.184067 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:58.184114 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:58.184182 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:58.185302 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:58.185898 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:58.187119 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:58.187465 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:58.187566 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:58.190248 (+ 2682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:58.190252 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:58.190254 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:58.190282 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:58.191632 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:58.192128 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:58.192132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:58.192134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:58.192213 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:58.192541 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:58.192545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:58.192546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:58.192741 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:58.192827 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:58.193341 (+ 514us) server_negotiation.cc:299] Negotiation successful
0812 19:03:58.193507 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 19:03:58.194487 31274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:58.181975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:44009 (local address 127.25.130.3:39559)
0812 19:03:58.182504 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:58.182582 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:58.183879 (+ 1297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:58.184323 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:58.184329 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:58.184695 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:58.185190 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:58.185204 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:58.186029 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:58.186031 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:58.187002 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:58.187013 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:58.187672 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:58.187678 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:58.187898 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:58.188510 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:58.188526 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:58.190059 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:58.191747 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:58.191754 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:58.191772 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:58.192027 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:58.192298 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:58.192301 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:58.192303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:58.192449 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:58.192845 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:58.192851 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:58.193101 (+ 250us) client_negotiation.cc:769] Sending connection context
0812 19:03:58.193271 (+ 170us) client_negotiation.cc:240] Negotiation successful
0812 19:03:58.193648 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":139,"threads_started":1}
I20260812 19:03:58.195604 31272 heartbeater.cc:347] Connected to a master server at 127.25.130.62:44009
I20260812 19:03:58.195832 31272 heartbeater.cc:464] Registering TS with master...
I20260812 19:03:58.196313 31272 heartbeater.cc:511] Master 127.25.130.62:44009 requested a full tablet report, sending...
I20260812 19:03:58.197300 30676 ts_manager.cc:194] Registered new tserver with Master: 8582f752cbd74045b4388a1014e00e5d (127.25.130.3:42349)
I20260812 19:03:58.197887 30676 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.3@KRBTEST.COM'} at 127.25.130.3:39559
I20260812 19:03:58.206565 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 19:03:58 dist-test-slave-49h3 krb5kdc[30616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 19:03:58 dist-test-slave-49h3 krb5kdc[30616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.1@KRBTEST.COM
I20260812 19:03:58.233942 31279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:58.224582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50970 (local address 127.25.130.1:35713)
0812 19:03:58.224957 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:58.224964 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:58.224990 (+ 26us) server_negotiation.cc:407] Connection header received
0812 19:03:58.225043 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:58.225047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:58.225243 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:58.225368 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:58.226115 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:58.226889 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:58.227726 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:58.228107 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:58.228187 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:58.230557 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:58.230571 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:58.230588 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:58.230617 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:58.232243 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:58.232678 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:58.232683 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:58.232684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:58.232768 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:58.232973 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:58.232974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:58.232975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:58.233234 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:58.233410 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:58.233645 (+ 235us) server_negotiation.cc:299] Negotiation successful
0812 19:03:58.233791 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":156,"threads_started":1}
Aug 12 19:03:58 dist-test-slave-49h3 krb5kdc[30616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:58.245530 30722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:58.236933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46918 (local address 127.25.130.62:44009)
0812 19:03:58.237091 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:58.237095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:58.237110 (+ 15us) server_negotiation.cc:407] Connection header received
0812 19:03:58.237270 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:58.237273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:58.237320 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:58.237387 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:58.238076 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:58.238649 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:58.239439 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:58.239660 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:58.239740 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:58.242273 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:58.242277 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:58.242279 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:58.242309 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:58.243674 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:58.244165 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:58.244170 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:58.244172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:58.244278 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:58.244551 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:58.244555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:58.244556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:58.244825 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:58.244953 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:58.245207 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 19:03:58.245334 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260812 19:03:58.245994 30676 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46918
I20260812 19:03:58.246678 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 30754
I20260812 19:03:58.256405 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 30922
I20260812 19:03:58.269295 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 31109
I20260812 19:03:58.280988 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 30633
2026-08-12T19:03:58Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2264 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:03:58 dist-test-slave-49h3 krb5kdc[31286](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:03:58 dist-test-slave-49h3 krb5kdc[31286](info): set up 2 sockets
Aug 12 19:03:58 dist-test-slave-49h3 krb5kdc[31286](info): commencing operation
krb5kdc: starting...
W20260812 19:03:59.342657 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:03:59 dist-test-slave-49h3 krb5kdc[31286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:03:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:03:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:03:59.474012 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:43881
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:43881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:59.589335 31303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:59.589684 31303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:59.589738 31303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:59.593963 31303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:59.594041 31303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:59.594060 31303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:59.594075 31303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:59.594090 31303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:59.599654 31303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:43881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:43881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:59.600833 31303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:59.601933 31303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:59.610706 31314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:59.610709 31309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:59.610831 31303 server_base.cc:1034] running on GCE node
W20260812 19:03:59.610709 31310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:59.611923 31303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:59.613220 31303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:59.614403 31303 hybrid_clock.cc:648] HybridClock initialized: now 1786561439614375 us; error 42 us; skew 500 ppm
Aug 12 19:03:59 dist-test-slave-49h3 krb5kdc[31286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561439, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:59.617282 31303 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:03:59.618148 31303 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:59.618227 31303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:59.618438 31303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:59.620649 31303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "c1601d85d326496a9ffd4bc9cff0be94"
format_stamp: "Formatted at 2026-08-12 19:03:59 on dist-test-slave-49h3"
server_key: "742876df4ba0c374570f2e690164595a"
server_key_iv: "0e4814cd75a055f38371e579bf3f02f2"
server_key_version: "encryptionkey@0"
I20260812 19:03:59.621274 31303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "c1601d85d326496a9ffd4bc9cff0be94"
format_stamp: "Formatted at 2026-08-12 19:03:59 on dist-test-slave-49h3"
server_key: "742876df4ba0c374570f2e690164595a"
server_key_iv: "0e4814cd75a055f38371e579bf3f02f2"
server_key_version: "encryptionkey@0"
I20260812 19:03:59.625335 31303 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260812 19:03:59.628013 31320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:59.629293 31303 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:03:59.629429 31303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "c1601d85d326496a9ffd4bc9cff0be94"
format_stamp: "Formatted at 2026-08-12 19:03:59 on dist-test-slave-49h3"
server_key: "742876df4ba0c374570f2e690164595a"
server_key_iv: "0e4814cd75a055f38371e579bf3f02f2"
server_key_version: "encryptionkey@0"
I20260812 19:03:59.629590 31303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:59.641373 31303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:59.644402 31303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:59.644611 31303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:59.652232 31303 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:43881
I20260812 19:03:59.652475 31384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:43881 every 8 connection(s)
I20260812 19:03:59.652959 31303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:59.655781 31385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:59.661904 31385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1601d85d326496a9ffd4bc9cff0be94: Bootstrap starting.
I20260812 19:03:59.662007 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 31303
I20260812 19:03:59.662132 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:03:59.662421 26120 external_mini_cluster.cc:1428] Setting key 5e025cf5618ae95e7d2504432b4e7370
I20260812 19:03:59.664839 31385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1601d85d326496a9ffd4bc9cff0be94: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:59.665709 31385 log.cc:824] T 00000000000000000000000000000000 P c1601d85d326496a9ffd4bc9cff0be94: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:59.667677 31385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1601d85d326496a9ffd4bc9cff0be94: No bootstrap required, opened a new log
Aug 12 19:03:59 dist-test-slave-49h3 krb5kdc[31286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:03:59.671659 31385 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1601d85d326496a9ffd4bc9cff0be94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1601d85d326496a9ffd4bc9cff0be94" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 43881 } }
I20260812 19:03:59.671911 31385 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1601d85d326496a9ffd4bc9cff0be94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:59.671958 31385 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1601d85d326496a9ffd4bc9cff0be94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1601d85d326496a9ffd4bc9cff0be94, State: Initialized, Role: FOLLOWER
I20260812 19:03:59.672444 31385 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1601d85d326496a9ffd4bc9cff0be94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1601d85d326496a9ffd4bc9cff0be94" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 43881 } }
I20260812 19:03:59.672617 31385 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1601d85d326496a9ffd4bc9cff0be94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:03:59.672690 31385 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1601d85d326496a9ffd4bc9cff0be94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:03:59.672838 31385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1601d85d326496a9ffd4bc9cff0be94 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:59.673916 31385 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1601d85d326496a9ffd4bc9cff0be94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1601d85d326496a9ffd4bc9cff0be94" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 43881 } }
I20260812 19:03:59.674276 31385 leader_election.cc:302] T 00000000000000000000000000000000 P c1601d85d326496a9ffd4bc9cff0be94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1601d85d326496a9ffd4bc9cff0be94; no voters:
I20260812 19:03:59.674606 31385 leader_election.cc:288] T 00000000000000000000000000000000 P c1601d85d326496a9ffd4bc9cff0be94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:03:59.674752 31390 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1601d85d326496a9ffd4bc9cff0be94 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:59.674995 31390 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1601d85d326496a9ffd4bc9cff0be94 [term 1 LEADER]: Becoming Leader. State: Replica: c1601d85d326496a9ffd4bc9cff0be94, State: Running, Role: LEADER
I20260812 19:03:59.675396 31390 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1601d85d326496a9ffd4bc9cff0be94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1601d85d326496a9ffd4bc9cff0be94" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 43881 } }
I20260812 19:03:59.675873 31385 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1601d85d326496a9ffd4bc9cff0be94 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:59.676748 31390 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1601d85d326496a9ffd4bc9cff0be94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1601d85d326496a9ffd4bc9cff0be94. Latest consensus state: current_term: 1 leader_uuid: "c1601d85d326496a9ffd4bc9cff0be94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1601d85d326496a9ffd4bc9cff0be94" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 43881 } } }
I20260812 19:03:59.676962 31390 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1601d85d326496a9ffd4bc9cff0be94 [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:59.677151 31391 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1601d85d326496a9ffd4bc9cff0be94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1601d85d326496a9ffd4bc9cff0be94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1601d85d326496a9ffd4bc9cff0be94" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 43881 } } }
I20260812 19:03:59.677255 31391 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1601d85d326496a9ffd4bc9cff0be94 [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:59.677412 31400 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:59.678148 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:59.663941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38004 (local address 127.25.130.62:43881)
0812 19:03:59.664434 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:59.664451 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:59.664490 (+ 39us) server_negotiation.cc:407] Connection header received
0812 19:03:59.665256 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:59.665277 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:59.665685 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:59.666065 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:59.666886 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:59.667911 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:59.668783 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:59.669276 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:59.669426 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:59.672138 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:59.672144 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:59.672160 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:59.672193 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:59.673971 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:59.674474 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:59.674480 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:59.674485 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:59.674653 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:59.675018 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:59.675022 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:59.675024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:59.675595 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:59.676531 (+ 936us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:59.676694 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 19:03:59.677027 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":162,"threads_started":1}
I20260812 19:03:59.680964 31400 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:03:59.688187 31400 catalog_manager.cc:1347] Generated new cluster ID: 57266e174b2143a8baf10520637bf55b
I20260812 19:03:59.688271 31400 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:59.700429 31400 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:59.701411 31400 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:59.707195 31400 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1601d85d326496a9ffd4bc9cff0be94: Generated new TSK 0
I20260812 19:03:59.708129 31400 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
I20260812 19:03:59.744583 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:0
--local_ip_for_outbound_sockets=127.25.130.1
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:43881
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:59.861800 31416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:59.862094 31416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:59.862140 31416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:59.866308 31416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:59.866389 31416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:59.866453 31416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:03:59.871963 31416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:43881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:59.873039 31416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:59.874146 31416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:59.884339 31425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:59.884339 31424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:59.884402 31416 server_base.cc:1034] running on GCE node
W20260812 19:03:59.884339 31427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:59.885066 31416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:59.885828 31416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:59.887001 31416 hybrid_clock.cc:648] HybridClock initialized: now 1786561439886982 us; error 30 us; skew 500 ppm
Aug 12 19:03:59 dist-test-slave-49h3 krb5kdc[31286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561439, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:59.890036 31416 init.cc:377] Logged in from keytab as kudu/127.25.130.1@KRBTEST.COM (short username kudu)
I20260812 19:03:59.890853 31416 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:59.890929 31416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:59.891176 31416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:59.893378 31416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-0/data/instance:
uuid: "29117594bcaf476290011190f97159a8"
format_stamp: "Formatted at 2026-08-12 19:03:59 on dist-test-slave-49h3"
server_key: "b86fad4e2d773ce6e59f5414565d4093"
server_key_iv: "4eaf7f061a82b91b207efb7a2c84472d"
server_key_version: "encryptionkey@0"
I20260812 19:03:59.894021 31416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance:
uuid: "29117594bcaf476290011190f97159a8"
format_stamp: "Formatted at 2026-08-12 19:03:59 on dist-test-slave-49h3"
server_key: "b86fad4e2d773ce6e59f5414565d4093"
server_key_iv: "4eaf7f061a82b91b207efb7a2c84472d"
server_key_version: "encryptionkey@0"
I20260812 19:03:59.897975 31416 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 19:03:59.900624 31433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:59.902082 31416 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:03:59.902215 31416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "29117594bcaf476290011190f97159a8"
format_stamp: "Formatted at 2026-08-12 19:03:59 on dist-test-slave-49h3"
server_key: "b86fad4e2d773ce6e59f5414565d4093"
server_key_iv: "4eaf7f061a82b91b207efb7a2c84472d"
server_key_version: "encryptionkey@0"
I20260812 19:03:59.902336 31416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:59.933696 31416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:59.937845 31416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:59.938066 31416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:59.938799 31416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:03:59.939929 31416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:03:59.939992 31416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:59.940059 31416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:03:59.940088 31416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:59.949048 31416 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:32867
I20260812 19:03:59.949057 31572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:32867 every 8 connection(s)
I20260812 19:03:59.949719 31416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:03:59.952311 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 31416
I20260812 19:03:59.952438 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance
I20260812 19:03:59.952754 26120 external_mini_cluster.cc:1428] Setting key 92458764075d16cccfb57e3e7c776ab9
Aug 12 19:03:59 dist-test-slave-49h3 krb5kdc[31286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561439, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.130.2@KRBTEST.COM; defaulting to no policy
I20260812 19:03:59.964309 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:59.951503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:43393 (local address 127.25.130.62:43881)
0812 19:03:59.951655 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:59.951660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:59.952625 (+ 965us) server_negotiation.cc:407] Connection header received
0812 19:03:59.954127 (+ 1502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:59.954131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:59.954187 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:59.954260 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:59.955671 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:59.956239 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:59.957501 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:59.957760 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:59.957861 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:59.960553 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:59.960556 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:59.960558 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:59.960588 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:59.962008 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:59.962590 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:59.962594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:59.962595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:59.962680 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:59.963129 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:59.963131 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:59.963132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:59.963328 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:59.963406 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:59.964040 (+ 634us) server_negotiation.cc:299] Negotiation successful
0812 19:03:59.964176 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 19:03:59.965265 31578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:59.951933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43881 (local address 127.25.130.1:43393)
0812 19:03:59.952448 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:59.952514 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:59.953898 (+ 1384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:59.954475 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:59.954485 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:59.954912 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:59.955459 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:59.955474 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:59.956388 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:59.956391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:59.957330 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:59.957341 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:59.957935 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:59.957939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:59.958128 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:59.958744 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:59.958762 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:59.960341 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:59.962153 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:59.962161 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:59.962182 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:59.962480 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:59.962802 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:59.962806 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:59.962808 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:59.963030 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:59.963421 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:59.963427 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:59.963796 (+ 369us) client_negotiation.cc:769] Sending connection context
0812 19:03:59.963974 (+ 178us) client_negotiation.cc:240] Negotiation successful
0812 19:03:59.964271 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":149,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.2@KRBTEST.COM" created.
I20260812 19:03:59.966836 31575 heartbeater.cc:347] Connected to a master server at 127.25.130.62:43881
I20260812 19:03:59.967135 31575 heartbeater.cc:464] Registering TS with master...
I20260812 19:03:59.967716 31575 heartbeater.cc:511] Master 127.25.130.62:43881 requested a full tablet report, sending...
I20260812 19:03:59.969429 31341 ts_manager.cc:194] Registered new tserver with Master: 29117594bcaf476290011190f97159a8 (127.25.130.1:32867)
I20260812 19:03:59.970688 31341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.1@KRBTEST.COM'} at 127.25.130.1:43393
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
I20260812 19:03:59.984303 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:0
--local_ip_for_outbound_sockets=127.25.130.2
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:43881
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:04:00.098253 31581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:00.098563 31581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:00.098639 31581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:00.102861 31581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:00.102941 31581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:00.103030 31581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:04:00.108829 31581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:43881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:00.110071 31581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:00.111195 31581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:00.121198 31587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:00.121196 31588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:00.121188 31591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:00.121603 31581 server_base.cc:1034] running on GCE node
I20260812 19:04:00.122085 31581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:00.122772 31581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:00.123970 31581 hybrid_clock.cc:648] HybridClock initialized: now 1786561440123944 us; error 38 us; skew 500 ppm
Aug 12 19:04:00 dist-test-slave-49h3 krb5kdc[31286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561440, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:00.126917 31581 init.cc:377] Logged in from keytab as kudu/127.25.130.2@KRBTEST.COM (short username kudu)
I20260812 19:04:00.127700 31581 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:00.127770 31581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:00.127991 31581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:00.130321 31581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-1/data/instance:
uuid: "0eed3b7951bd418585e0d7eab10e1896"
format_stamp: "Formatted at 2026-08-12 19:04:00 on dist-test-slave-49h3"
server_key: "25bed7216033597b0c5c4539d21d92ca"
server_key_iv: "c5a9ecf6a80e38ca8ebb7eb8512fd9af"
server_key_version: "encryptionkey@0"
I20260812 19:04:00.130909 31581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance:
uuid: "0eed3b7951bd418585e0d7eab10e1896"
format_stamp: "Formatted at 2026-08-12 19:04:00 on dist-test-slave-49h3"
server_key: "25bed7216033597b0c5c4539d21d92ca"
server_key_iv: "c5a9ecf6a80e38ca8ebb7eb8512fd9af"
server_key_version: "encryptionkey@0"
I20260812 19:04:00.134927 31581 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:04:00.137507 31597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:00.138821 31581 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:04:00.138934 31581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "0eed3b7951bd418585e0d7eab10e1896"
format_stamp: "Formatted at 2026-08-12 19:04:00 on dist-test-slave-49h3"
server_key: "25bed7216033597b0c5c4539d21d92ca"
server_key_iv: "c5a9ecf6a80e38ca8ebb7eb8512fd9af"
server_key_version: "encryptionkey@0"
I20260812 19:04:00.139060 31581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:00.155917 31581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:00.159036 31581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:00.159233 31581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:00.159879 31581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:00.160774 31581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:00.160825 31581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:00.160871 31581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:00.160899 31581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:00.169991 31581 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:35897
I20260812 19:04:00.170037 31739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:35897 every 8 connection(s)
I20260812 19:04:00.170646 31581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:04:00.171840 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 31581
I20260812 19:04:00.171975 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance
I20260812 19:04:00.172324 26120 external_mini_cluster.cc:1428] Setting key 0f94fd0b4a19735126766f13f837b8e0
Aug 12 19:04:00 dist-test-slave-49h3 krb5kdc[31286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561440, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.130.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.3@KRBTEST.COM" created.
I20260812 19:04:00.187467 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:00.173590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:33591 (local address 127.25.130.62:43881)
0812 19:04:00.173737 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:00.173741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:00.174626 (+ 885us) server_negotiation.cc:407] Connection header received
0812 19:04:00.176382 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:00.176386 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:00.176451 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:00.176560 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:00.178186 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.178723 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.179966 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.180199 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.180296 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:00.183401 (+ 3105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:00.183406 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:00.183408 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:00.183440 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:00.184990 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:00.185628 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:00.185631 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:00.185633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:00.185728 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:00.186154 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:00.186157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:00.186158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:00.186412 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:00.186505 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:00.187143 (+ 638us) server_negotiation.cc:299] Negotiation successful
0812 19:04:00.187288 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 19:04:00.188412 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:00.173850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43881 (local address 127.25.130.2:33591)
0812 19:04:00.174427 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:00.174498 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:00.176047 (+ 1549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:00.176802 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:00.176814 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:00.177333 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:00.178007 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:00.178024 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.178885 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.178888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:00.179825 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:00.179836 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.180369 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.180373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:00.180552 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:00.181230 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:00.181248 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:00.183022 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:00.185141 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:00.185149 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:00.185167 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:00.185512 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:00.185826 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:00.185830 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:00.185832 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:00.185999 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:00.186535 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:00.186543 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:00.186869 (+ 326us) client_negotiation.cc:769] Sending connection context
0812 19:04:00.187072 (+ 203us) client_negotiation.cc:240] Negotiation successful
0812 19:04:00.187469 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":143,"threads_started":1}
I20260812 19:04:00.189720 31740 heartbeater.cc:347] Connected to a master server at 127.25.130.62:43881
I20260812 19:04:00.190032 31740 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:00.190604 31740 heartbeater.cc:511] Master 127.25.130.62:43881 requested a full tablet report, sending...
I20260812 19:04:00.191862 31341 ts_manager.cc:194] Registered new tserver with Master: 0eed3b7951bd418585e0d7eab10e1896 (127.25.130.2:35897)
I20260812 19:04:00.192673 31341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.2@KRBTEST.COM'} at 127.25.130.2:33591
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
I20260812 19:04:00.204977 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:0
--local_ip_for_outbound_sockets=127.25.130.3
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:43881
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:04:00.321785 31745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:00.322084 31745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:00.322132 31745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:00.326371 31745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:00.326459 31745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:00.326524 31745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:04:00.332280 31745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:43881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:00.333554 31745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:00.334684 31745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:00.344990 31753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:00.345101 31745 server_base.cc:1034] running on GCE node
W20260812 19:04:00.345014 31750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:00.344990 31751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:00.345976 31745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:00.346722 31745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:00.347940 31745 hybrid_clock.cc:648] HybridClock initialized: now 1786561440347907 us; error 59 us; skew 500 ppm
Aug 12 19:04:00 dist-test-slave-49h3 krb5kdc[31286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561440, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:00.351105 31745 init.cc:377] Logged in from keytab as kudu/127.25.130.3@KRBTEST.COM (short username kudu)
I20260812 19:04:00.351958 31745 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:00.352042 31745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:00.352383 31745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:00.354719 31745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-2/data/instance:
uuid: "e4a95228fe5d4a8fa48909901b6bb225"
format_stamp: "Formatted at 2026-08-12 19:04:00 on dist-test-slave-49h3"
server_key: "dd924d9c45ee5d8c54a02551057e0d2c"
server_key_iv: "165b9f94a44d416d6a9e5820a7e477a2"
server_key_version: "encryptionkey@0"
I20260812 19:04:00.355326 31745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance:
uuid: "e4a95228fe5d4a8fa48909901b6bb225"
format_stamp: "Formatted at 2026-08-12 19:04:00 on dist-test-slave-49h3"
server_key: "dd924d9c45ee5d8c54a02551057e0d2c"
server_key_iv: "165b9f94a44d416d6a9e5820a7e477a2"
server_key_version: "encryptionkey@0"
I20260812 19:04:00.359460 31745 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:04:00.362164 31762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:00.363565 31745 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:04:00.363703 31745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "e4a95228fe5d4a8fa48909901b6bb225"
format_stamp: "Formatted at 2026-08-12 19:04:00 on dist-test-slave-49h3"
server_key: "dd924d9c45ee5d8c54a02551057e0d2c"
server_key_iv: "165b9f94a44d416d6a9e5820a7e477a2"
server_key_version: "encryptionkey@0"
I20260812 19:04:00.363824 31745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:00.374157 31745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:00.377562 31745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:00.377759 31745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:00.378393 31745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:00.379303 31745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:00.379356 31745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:00.379412 31745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:00.379439 31745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:00.388615 31745 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:37617
I20260812 19:04:00.388652 31901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:37617 every 8 connection(s)
I20260812 19:04:00.389266 31745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:04:00.393328 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 31745
I20260812 19:04:00.393488 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance
I20260812 19:04:00.393805 26120 external_mini_cluster.cc:1428] Setting key f7b867b66fc477a67e8a0f7b2f542706
Aug 12 19:04:00 dist-test-slave-49h3 krb5kdc[31286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561440, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:04:00.403465 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:00.391163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:48949 (local address 127.25.130.62:43881)
0812 19:04:00.391318 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:00.391322 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:00.392270 (+ 948us) server_negotiation.cc:407] Connection header received
0812 19:04:00.393669 (+ 1399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:00.393673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:00.393738 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:00.393838 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:00.395349 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.395877 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.397043 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.397268 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.397357 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:00.399926 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:00.399929 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:00.399931 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:00.399957 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:00.401235 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:00.401839 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:00.401842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:00.401845 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:00.401930 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:00.402293 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:00.402296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:00.402297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:00.402498 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:00.402592 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:00.403154 (+ 562us) server_negotiation.cc:299] Negotiation successful
0812 19:04:00.403291 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 19:04:00.404335 31905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:00.391597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43881 (local address 127.25.130.3:48949)
0812 19:04:00.392098 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:00.392152 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:00.393394 (+ 1242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:00.394018 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:00.394026 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:00.394502 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:00.395191 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:00.395215 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.396011 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.396015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:00.396910 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:00.396920 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.397417 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.397421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:00.397612 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:00.398201 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:00.398219 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:00.399732 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:00.401366 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:00.401373 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:00.401392 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:00.401731 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:00.402023 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:00.402026 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:00.402028 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:00.402185 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:00.402609 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:00.402617 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:00.402886 (+ 269us) client_negotiation.cc:769] Sending connection context
0812 19:04:00.403065 (+ 179us) client_negotiation.cc:240] Negotiation successful
0812 19:04:00.403433 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":135,"threads_started":1}
I20260812 19:04:00.405517 31902 heartbeater.cc:347] Connected to a master server at 127.25.130.62:43881
I20260812 19:04:00.405743 31902 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:00.406243 31902 heartbeater.cc:511] Master 127.25.130.62:43881 requested a full tablet report, sending...
I20260812 19:04:00.407459 31341 ts_manager.cc:194] Registered new tserver with Master: e4a95228fe5d4a8fa48909901b6bb225 (127.25.130.3:37617)
I20260812 19:04:00.408072 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:04:00.408171 31341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.3@KRBTEST.COM'} at 127.25.130.3:48949
I20260812 19:04:00.417021 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:00.410190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38020 (local address 127.25.130.62:43881)
0812 19:04:00.410309 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:00.410313 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:00.410482 (+ 169us) server_negotiation.cc:407] Connection header received
0812 19:04:00.410589 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:00.410591 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:00.410629 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:00.410703 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:00.411390 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.411930 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.412703 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.412902 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.412970 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:00.414317 (+ 1347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:00.414320 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:00.414322 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:00.414347 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:00.415510 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:00.415914 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:00.415917 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:00.415918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:00.415989 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:00.416258 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:00.416260 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:00.416261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:00.416495 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:00.416598 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:00.416788 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 19:04:00.416890 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 19:04:00.420110 31341 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:04:00.423545 31341 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:04:00.439810 31918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:00.433089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55890 (local address 127.25.130.3:37617)
0812 19:04:00.434042 (+ 953us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:00.434055 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:00.434084 (+ 29us) server_negotiation.cc:407] Connection header received
0812 19:04:00.434184 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:00.434190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:00.434428 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:00.434978 (+ 550us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:00.435148 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.436367 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.438143 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.439021 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.439151 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:00.439340 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:00.439504 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 19:04:00.439616 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":835,"thread_start_us":207,"threads_started":1}
I20260812 19:04:00.439806 31917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:00.432896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:37617 (local address 127.0.0.1:55890)
0812 19:04:00.433320 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:00.433377 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:00.433591 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:00.434606 (+ 1015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:00.434613 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:00.434637 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:00.434832 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:00.434839 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.436537 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.436541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:00.438001 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:00.438012 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.439162 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.439166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:00.439272 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:00.439323 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 19:04:00.439452 (+ 129us) client_negotiation.cc:240] Negotiation successful
0812 19:04:00.439625 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":138,"thread_start_us":80,"threads_started":1}
I20260812 19:04:00.439851 31924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:00.434186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50100 (local address 127.25.130.1:32867)
0812 19:04:00.434468 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:00.434475 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:00.434500 (+ 25us) server_negotiation.cc:407] Connection header received
0812 19:04:00.434579 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:00.434584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:00.434791 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:00.434966 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:00.435306 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.436410 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.438363 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.439178 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.439283 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:00.439406 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:00.439598 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 19:04:00.439704 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":78,"threads_started":1}
I20260812 19:04:00.440330 31922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:00.433301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:32867 (local address 127.0.0.1:50100)
0812 19:04:00.433881 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:00.433899 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:00.433991 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:00.435050 (+ 1059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:00.435053 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:00.435072 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:00.435184 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:00.435190 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.436557 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.436561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:00.438220 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:00.438233 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.439319 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.439322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:00.439400 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:00.439422 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 19:04:00.439481 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 19:04:00.440184 (+ 703us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"mutex_wait_us":758,"thread_start_us":103,"threads_started":1}
I20260812 19:04:00.441581 31923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:00.433815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:35897 (local address 127.0.0.1:49306)
0812 19:04:00.434087 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:00.434101 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:00.434187 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:00.435474 (+ 1287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:00.435477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:00.435494 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:00.435608 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:00.435614 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.436632 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.436635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:00.438221 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:00.438233 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.441106 (+ 2873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.441110 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:00.441231 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:00.441261 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 19:04:00.441316 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 19:04:00.441397 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":66,"thread_start_us":126,"threads_started":1}
I20260812 19:04:00.442332 31925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:00.434694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49306 (local address 127.25.130.2:35897)
0812 19:04:00.435025 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:00.435043 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:00.435065 (+ 22us) server_negotiation.cc:407] Connection header received
0812 19:04:00.435154 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:00.435160 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:00.435348 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:00.435478 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:00.435731 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.436493 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.439484 (+ 2991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.440957 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.441880 (+ 923us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:00.442046 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:00.442106 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 19:04:00.442208 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":130,"threads_started":1}
I20260812 19:04:00.443449 31823 tablet_service.cc:1467] Processing CreateTablet for tablet 0e036f44c82f4fbdae5706eef01ec5d7 (DEFAULT_TABLE table=test-table [id=019b9d55504a47848061d3fbb26a0ba9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:04:00.443446 31493 tablet_service.cc:1467] Processing CreateTablet for tablet 0e036f44c82f4fbdae5706eef01ec5d7 (DEFAULT_TABLE table=test-table [id=019b9d55504a47848061d3fbb26a0ba9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:04:00.444720 31823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e036f44c82f4fbdae5706eef01ec5d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:00.444717 31493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e036f44c82f4fbdae5706eef01ec5d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:00.444952 31660 tablet_service.cc:1467] Processing CreateTablet for tablet 0e036f44c82f4fbdae5706eef01ec5d7 (DEFAULT_TABLE table=test-table [id=019b9d55504a47848061d3fbb26a0ba9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:04:00.446185 31660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e036f44c82f4fbdae5706eef01ec5d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:00.452585 31926 tablet_bootstrap.cc:492] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896: Bootstrap starting.
I20260812 19:04:00.453894 31929 tablet_bootstrap.cc:492] T 0e036f44c82f4fbdae5706eef01ec5d7 P e4a95228fe5d4a8fa48909901b6bb225: Bootstrap starting.
I20260812 19:04:00.453997 31927 tablet_bootstrap.cc:492] T 0e036f44c82f4fbdae5706eef01ec5d7 P 29117594bcaf476290011190f97159a8: Bootstrap starting.
I20260812 19:04:00.455317 31926 tablet_bootstrap.cc:654] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:00.455855 31927 tablet_bootstrap.cc:654] T 0e036f44c82f4fbdae5706eef01ec5d7 P 29117594bcaf476290011190f97159a8: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:00.456123 31929 tablet_bootstrap.cc:654] T 0e036f44c82f4fbdae5706eef01ec5d7 P e4a95228fe5d4a8fa48909901b6bb225: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:00.456753 31926 log.cc:824] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:00.457115 31927 log.cc:824] T 0e036f44c82f4fbdae5706eef01ec5d7 P 29117594bcaf476290011190f97159a8: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:00.457314 31929 log.cc:824] T 0e036f44c82f4fbdae5706eef01ec5d7 P e4a95228fe5d4a8fa48909901b6bb225: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:00.459132 31929 tablet_bootstrap.cc:492] T 0e036f44c82f4fbdae5706eef01ec5d7 P e4a95228fe5d4a8fa48909901b6bb225: No bootstrap required, opened a new log
I20260812 19:04:00.459167 31927 tablet_bootstrap.cc:492] T 0e036f44c82f4fbdae5706eef01ec5d7 P 29117594bcaf476290011190f97159a8: No bootstrap required, opened a new log
I20260812 19:04:00.459173 31926 tablet_bootstrap.cc:492] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896: No bootstrap required, opened a new log
I20260812 19:04:00.459365 31929 ts_tablet_manager.cc:1397] T 0e036f44c82f4fbdae5706eef01ec5d7 P e4a95228fe5d4a8fa48909901b6bb225: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 19:04:00.459365 31927 ts_tablet_manager.cc:1397] T 0e036f44c82f4fbdae5706eef01ec5d7 P 29117594bcaf476290011190f97159a8: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260812 19:04:00.459365 31926 ts_tablet_manager.cc:1397] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 19:04:00.462529 31929 raft_consensus.cc:348] T 0e036f44c82f4fbdae5706eef01ec5d7 P e4a95228fe5d4a8fa48909901b6bb225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a95228fe5d4a8fa48909901b6bb225" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 37617 } } peers { permanent_uuid: "0eed3b7951bd418585e0d7eab10e1896" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 35897 } } peers { permanent_uuid: "29117594bcaf476290011190f97159a8" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32867 } }
I20260812 19:04:00.462813 31929 raft_consensus.cc:374] T 0e036f44c82f4fbdae5706eef01ec5d7 P e4a95228fe5d4a8fa48909901b6bb225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:00.462870 31929 raft_consensus.cc:729] T 0e036f44c82f4fbdae5706eef01ec5d7 P e4a95228fe5d4a8fa48909901b6bb225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4a95228fe5d4a8fa48909901b6bb225, State: Initialized, Role: FOLLOWER
I20260812 19:04:00.463369 31929 consensus_queue.cc:260] T 0e036f44c82f4fbdae5706eef01ec5d7 P e4a95228fe5d4a8fa48909901b6bb225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a95228fe5d4a8fa48909901b6bb225" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 37617 } } peers { permanent_uuid: "0eed3b7951bd418585e0d7eab10e1896" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 35897 } } peers { permanent_uuid: "29117594bcaf476290011190f97159a8" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32867 } }
I20260812 19:04:00.463295 31927 raft_consensus.cc:348] T 0e036f44c82f4fbdae5706eef01ec5d7 P 29117594bcaf476290011190f97159a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a95228fe5d4a8fa48909901b6bb225" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 37617 } } peers { permanent_uuid: "0eed3b7951bd418585e0d7eab10e1896" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 35897 } } peers { permanent_uuid: "29117594bcaf476290011190f97159a8" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32867 } }
I20260812 19:04:00.463295 31926 raft_consensus.cc:348] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a95228fe5d4a8fa48909901b6bb225" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 37617 } } peers { permanent_uuid: "0eed3b7951bd418585e0d7eab10e1896" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 35897 } } peers { permanent_uuid: "29117594bcaf476290011190f97159a8" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32867 } }
I20260812 19:04:00.463562 31926 raft_consensus.cc:374] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:00.463562 31927 raft_consensus.cc:374] T 0e036f44c82f4fbdae5706eef01ec5d7 P 29117594bcaf476290011190f97159a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:00.463613 31926 raft_consensus.cc:729] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0eed3b7951bd418585e0d7eab10e1896, State: Initialized, Role: FOLLOWER
I20260812 19:04:00.463624 31927 raft_consensus.cc:729] T 0e036f44c82f4fbdae5706eef01ec5d7 P 29117594bcaf476290011190f97159a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29117594bcaf476290011190f97159a8, State: Initialized, Role: FOLLOWER
I20260812 19:04:00.464067 31927 consensus_queue.cc:260] T 0e036f44c82f4fbdae5706eef01ec5d7 P 29117594bcaf476290011190f97159a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a95228fe5d4a8fa48909901b6bb225" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 37617 } } peers { permanent_uuid: "0eed3b7951bd418585e0d7eab10e1896" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 35897 } } peers { permanent_uuid: "29117594bcaf476290011190f97159a8" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32867 } }
I20260812 19:04:00.464102 31926 consensus_queue.cc:260] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a95228fe5d4a8fa48909901b6bb225" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 37617 } } peers { permanent_uuid: "0eed3b7951bd418585e0d7eab10e1896" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 35897 } } peers { permanent_uuid: "29117594bcaf476290011190f97159a8" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32867 } }
I20260812 19:04:00.464550 31929 ts_tablet_manager.cc:1428] T 0e036f44c82f4fbdae5706eef01ec5d7 P e4a95228fe5d4a8fa48909901b6bb225: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 19:04:00.464936 31902 heartbeater.cc:503] Master 127.25.130.62:43881 was elected leader, sending a full tablet report...
I20260812 19:04:00.465085 31926 ts_tablet_manager.cc:1428] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 19:04:00.465085 31927 ts_tablet_manager.cc:1428] T 0e036f44c82f4fbdae5706eef01ec5d7 P 29117594bcaf476290011190f97159a8: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260812 19:04:00.465464 31575 heartbeater.cc:503] Master 127.25.130.62:43881 was elected leader, sending a full tablet report...
I20260812 19:04:00.465464 31740 heartbeater.cc:503] Master 127.25.130.62:43881 was elected leader, sending a full tablet report...
I20260812 19:04:00.524574 31937 raft_consensus.cc:482] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:04:00.524830 31937 raft_consensus.cc:504] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a95228fe5d4a8fa48909901b6bb225" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 37617 } } peers { permanent_uuid: "0eed3b7951bd418585e0d7eab10e1896" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 35897 } } peers { permanent_uuid: "29117594bcaf476290011190f97159a8" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32867 } }
I20260812 19:04:00.526201 31937 leader_election.cc:288] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4a95228fe5d4a8fa48909901b6bb225 (127.25.130.3:37617), 29117594bcaf476290011190f97159a8 (127.25.130.1:32867)
I20260812 19:04:00.531549 31918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:00.526658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:34559 (local address 127.25.130.3:37617)
0812 19:04:00.526812 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:00.526816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:00.527095 (+ 279us) server_negotiation.cc:407] Connection header received
0812 19:04:00.527156 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:00.527159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:00.527205 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:00.527279 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:00.527503 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.528310 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.530219 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.530880 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.531028 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:00.531087 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:00.531316 (+ 229us) server_negotiation.cc:299] Negotiation successful
0812 19:04:00.531398 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 19:04:00.531549 31924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:00.526627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:47631 (local address 127.25.130.1:32867)
0812 19:04:00.526779 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:00.526783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:00.526861 (+ 78us) server_negotiation.cc:407] Connection header received
0812 19:04:00.527012 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:00.527014 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:00.527067 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:00.527170 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:00.527479 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.528295 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.530219 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.530879 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.531028 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:00.531087 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:00.531316 (+ 229us) server_negotiation.cc:299] Negotiation successful
0812 19:04:00.531399 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 19:04:00.531598 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:00.526507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:32867 (local address 127.25.130.2:47631)
0812 19:04:00.526777 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:00.526792 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:00.526901 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:00.527208 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:00.527210 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:00.527224 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:00.527353 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:00.527359 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.528420 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.528424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:00.530013 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:00.530031 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.531029 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.531033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:00.531206 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:00.531223 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 19:04:00.531288 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 19:04:00.531412 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":175}
I20260812 19:04:00.531605 31939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:00.526507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:37617 (local address 127.25.130.2:34559)
0812 19:04:00.526990 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:00.527002 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:00.527085 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:00.527296 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:00.527299 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:00.527312 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:00.527403 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:00.527409 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.528420 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.528424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:00.530013 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:00.530031 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.531029 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.531033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:00.531164 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:00.531202 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 19:04:00.531288 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 19:04:00.531428 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"mutex_wait_us":185,"thread_start_us":135,"threads_started":1}
I20260812 19:04:00.532634 31518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e036f44c82f4fbdae5706eef01ec5d7" candidate_uuid: "0eed3b7951bd418585e0d7eab10e1896" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29117594bcaf476290011190f97159a8" is_pre_election: true
I20260812 19:04:00.532660 31847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e036f44c82f4fbdae5706eef01ec5d7" candidate_uuid: "0eed3b7951bd418585e0d7eab10e1896" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4a95228fe5d4a8fa48909901b6bb225" is_pre_election: true
I20260812 19:04:00.533013 31847 raft_consensus.cc:2454] T 0e036f44c82f4fbdae5706eef01ec5d7 P e4a95228fe5d4a8fa48909901b6bb225 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0eed3b7951bd418585e0d7eab10e1896 in term 0.
I20260812 19:04:00.533013 31518 raft_consensus.cc:2454] T 0e036f44c82f4fbdae5706eef01ec5d7 P 29117594bcaf476290011190f97159a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0eed3b7951bd418585e0d7eab10e1896 in term 0.
I20260812 19:04:00.533609 31600 leader_election.cc:302] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0eed3b7951bd418585e0d7eab10e1896, 29117594bcaf476290011190f97159a8, e4a95228fe5d4a8fa48909901b6bb225; no voters:
I20260812 19:04:00.533938 31937 raft_consensus.cc:2781] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:04:00.534061 31937 raft_consensus.cc:482] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:04:00.534101 31937 raft_consensus.cc:3037] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:00.535257 31937 raft_consensus.cc:504] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a95228fe5d4a8fa48909901b6bb225" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 37617 } } peers { permanent_uuid: "0eed3b7951bd418585e0d7eab10e1896" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 35897 } } peers { permanent_uuid: "29117594bcaf476290011190f97159a8" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32867 } }
I20260812 19:04:00.535667 31937 leader_election.cc:288] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896 [CANDIDATE]: Term 1 election: Requested vote from peers e4a95228fe5d4a8fa48909901b6bb225 (127.25.130.3:37617), 29117594bcaf476290011190f97159a8 (127.25.130.1:32867)
I20260812 19:04:00.536119 31847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e036f44c82f4fbdae5706eef01ec5d7" candidate_uuid: "0eed3b7951bd418585e0d7eab10e1896" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4a95228fe5d4a8fa48909901b6bb225"
I20260812 19:04:00.536129 31518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e036f44c82f4fbdae5706eef01ec5d7" candidate_uuid: "0eed3b7951bd418585e0d7eab10e1896" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29117594bcaf476290011190f97159a8"
I20260812 19:04:00.536284 31847 raft_consensus.cc:3037] T 0e036f44c82f4fbdae5706eef01ec5d7 P e4a95228fe5d4a8fa48909901b6bb225 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:00.536283 31518 raft_consensus.cc:3037] T 0e036f44c82f4fbdae5706eef01ec5d7 P 29117594bcaf476290011190f97159a8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:00.537710 31518 raft_consensus.cc:2454] T 0e036f44c82f4fbdae5706eef01ec5d7 P 29117594bcaf476290011190f97159a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0eed3b7951bd418585e0d7eab10e1896 in term 1.
I20260812 19:04:00.537710 31847 raft_consensus.cc:2454] T 0e036f44c82f4fbdae5706eef01ec5d7 P e4a95228fe5d4a8fa48909901b6bb225 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0eed3b7951bd418585e0d7eab10e1896 in term 1.
I20260812 19:04:00.538085 31600 leader_election.cc:302] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0eed3b7951bd418585e0d7eab10e1896, 29117594bcaf476290011190f97159a8; no voters:
I20260812 19:04:00.538281 31937 raft_consensus.cc:2781] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:04:00.538530 31937 raft_consensus.cc:686] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896 [term 1 LEADER]: Becoming Leader. State: Replica: 0eed3b7951bd418585e0d7eab10e1896, State: Running, Role: LEADER
I20260812 19:04:00.538918 31937 consensus_queue.cc:237] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a95228fe5d4a8fa48909901b6bb225" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 37617 } } peers { permanent_uuid: "0eed3b7951bd418585e0d7eab10e1896" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 35897 } } peers { permanent_uuid: "29117594bcaf476290011190f97159a8" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32867 } }
I20260812 19:04:00.542424 31340 catalog_manager.cc:5495] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0eed3b7951bd418585e0d7eab10e1896 (127.25.130.2). New cstate: current_term: 1 leader_uuid: "0eed3b7951bd418585e0d7eab10e1896" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a95228fe5d4a8fa48909901b6bb225" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 37617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0eed3b7951bd418585e0d7eab10e1896" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 35897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29117594bcaf476290011190f97159a8" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32867 } health_report { overall_health: UNKNOWN } } }
I20260812 19:04:00.565913 31925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:00.562186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49312 (local address 127.25.130.2:35897)
0812 19:04:00.562330 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:00.562334 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:00.562363 (+ 29us) server_negotiation.cc:407] Connection header received
0812 19:04:00.562452 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:00.562454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:00.562512 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:00.562624 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:04:00.562926 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.563449 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.564338 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:00.564568 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:00.564660 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:00.564993 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:04:00.565473 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:04:00.565597 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:00.565670 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 19:04:00.565736 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 19:04:00.572446 31518 raft_consensus.cc:1261] T 0e036f44c82f4fbdae5706eef01ec5d7 P 29117594bcaf476290011190f97159a8 [term 1 FOLLOWER]: Refusing update from remote peer 0eed3b7951bd418585e0d7eab10e1896: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:04:00.572446 31847 raft_consensus.cc:1261] T 0e036f44c82f4fbdae5706eef01ec5d7 P e4a95228fe5d4a8fa48909901b6bb225 [term 1 FOLLOWER]: Refusing update from remote peer 0eed3b7951bd418585e0d7eab10e1896: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:04:00.573140 31937 consensus_queue.cc:1035] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29117594bcaf476290011190f97159a8" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:04:00.573359 31940 consensus_queue.cc:1035] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4a95228fe5d4a8fa48909901b6bb225" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 37617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:04:00.582070 31945 mvcc.cc:205] Tried to move back new op lower bound from 7317755660576055296 to 7317755660450205696. Current Snapshot: MvccSnapshot[applied={T|T < 7317755660576055296}]
I20260812 19:04:00.583067 31947 mvcc.cc:205] Tried to move back new op lower bound from 7317755660576055296 to 7317755660450205696. Current Snapshot: MvccSnapshot[applied={T|T < 7317755660576055296}]
I20260812 19:04:00.591667 31341 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:04:00.591892 31341 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:04:00.594774 31341 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c1601d85d326496a9ffd4bc9cff0be94: Sending DeleteTablet for 3 replicas of tablet 0e036f44c82f4fbdae5706eef01ec5d7
I20260812 19:04:00.595767 31823 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e036f44c82f4fbdae5706eef01ec5d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:04:00 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:55890
I20260812 19:04:00.595793 31660 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e036f44c82f4fbdae5706eef01ec5d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:04:00 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:49306
I20260812 19:04:00.595814 31493 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e036f44c82f4fbdae5706eef01ec5d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:04:00 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:50100
I20260812 19:04:00.596391 31958 tablet_replica.cc:330] T 0e036f44c82f4fbdae5706eef01ec5d7 P 29117594bcaf476290011190f97159a8: stopping tablet replica
I20260812 19:04:00.596671 31956 tablet_replica.cc:330] T 0e036f44c82f4fbdae5706eef01ec5d7 P e4a95228fe5d4a8fa48909901b6bb225: stopping tablet replica
I20260812 19:04:00.596803 31958 raft_consensus.cc:2229] T 0e036f44c82f4fbdae5706eef01ec5d7 P 29117594bcaf476290011190f97159a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:04:00.596927 31957 tablet_replica.cc:330] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896: stopping tablet replica
I20260812 19:04:00.597138 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 31416
I20260812 19:04:00.597182 31958 raft_consensus.cc:2258] T 0e036f44c82f4fbdae5706eef01ec5d7 P 29117594bcaf476290011190f97159a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:04:00.598052 31956 raft_consensus.cc:2229] T 0e036f44c82f4fbdae5706eef01ec5d7 P e4a95228fe5d4a8fa48909901b6bb225 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:04:00.598276 31957 raft_consensus.cc:2229] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896 [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:04:00.598690 31957 raft_consensus.cc:2258] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:04:00.599005 31956 raft_consensus.cc:2258] T 0e036f44c82f4fbdae5706eef01ec5d7 P e4a95228fe5d4a8fa48909901b6bb225 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:04:00.600792 31957 ts_tablet_manager.cc:1905] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:04:00.603679 31956 ts_tablet_manager.cc:1905] T 0e036f44c82f4fbdae5706eef01ec5d7 P e4a95228fe5d4a8fa48909901b6bb225: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:04:00.603911 31957 ts_tablet_manager.cc:1918] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:04:00.604106 31957 log.cc:1196] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-1/wal/wals/0e036f44c82f4fbdae5706eef01ec5d7
I20260812 19:04:00.604523 31957 ts_tablet_manager.cc:1939] T 0e036f44c82f4fbdae5706eef01ec5d7 P 0eed3b7951bd418585e0d7eab10e1896: Deleting consensus metadata
I20260812 19:04:00.606043 31326 catalog_manager.cc:4841] TS 0eed3b7951bd418585e0d7eab10e1896 (127.25.130.2:35897): tablet 0e036f44c82f4fbdae5706eef01ec5d7 (table test-table [id=019b9d55504a47848061d3fbb26a0ba9]) successfully deleted
I20260812 19:04:00.606523 31956 ts_tablet_manager.cc:1918] T 0e036f44c82f4fbdae5706eef01ec5d7 P e4a95228fe5d4a8fa48909901b6bb225: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:04:00.606644 31956 log.cc:1196] T 0e036f44c82f4fbdae5706eef01ec5d7 P e4a95228fe5d4a8fa48909901b6bb225: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561415988404-26120-0/minicluster-data/ts-2/wal/wals/0e036f44c82f4fbdae5706eef01ec5d7
I20260812 19:04:00.607019 31956 ts_tablet_manager.cc:1939] T 0e036f44c82f4fbdae5706eef01ec5d7 P e4a95228fe5d4a8fa48909901b6bb225: Deleting consensus metadata
I20260812 19:04:00.608592 31326 catalog_manager.cc:4841] TS e4a95228fe5d4a8fa48909901b6bb225 (127.25.130.3:37617): tablet 0e036f44c82f4fbdae5706eef01ec5d7 (table test-table [id=019b9d55504a47848061d3fbb26a0ba9]) successfully deleted
W20260812 19:04:00.608997 31323 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.130.1:32867 (error 108)
I20260812 19:04:00.609143 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 31581
I20260812 19:04:00.611738 31922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:00.611390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:32867 (local address 127.0.0.1:50116)
0812 19:04:00.611546 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:00.611634 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.1:32867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 19:04:00.611953 31323 catalog_manager.cc:4584] TS 29117594bcaf476290011190f97159a8 (127.25.130.1:32867): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0e036f44c82f4fbdae5706eef01ec5d7: Network error: Client connection negotiation failed: client connection to 127.25.130.1:32867: connect: Connection refused (error 111)
I20260812 19:04:00.617973 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 31745
I20260812 19:04:00.627260 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 31303
2026-08-12T19:04:00Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2348 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T19:04:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:04:00Z Disabled control of system clock
I20260812 19:04:00.664608 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:39193
--webserver_interface=127.25.130.62
--webserver_port=0
--builtin_ntp_servers=127.25.130.20:37319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:39193
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 19:04:00.783025 31963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:00.783339 31963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:00.783376 31963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:00.787523 31963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:04:00.787608 31963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:00.787623 31963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:00.787648 31963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:04:00.787667 31963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:04:00.792760 31963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:37319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:39193
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:39193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:00.794013 31963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:00.795483 31963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:00.804517 31971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:00.804517 31970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:00.804509 31974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:00.804791 31963 server_base.cc:1034] running on GCE node
I20260812 19:04:00.805540 31963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:00.806783 31963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:00.807960 31963 hybrid_clock.cc:648] HybridClock initialized: now 1786561440807931 us; error 45 us; skew 500 ppm
I20260812 19:04:00.810257 31963 webserver.cc:460] Webserver started at http://127.25.130.62:40481/ using document root <none> and password file <none>
I20260812 19:04:00.810907 31963 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:00.810982 31963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:00.811216 31963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:00.813639 31963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "2e2ee88466f64b3fb1b7f8e546a4036b"
format_stamp: "Formatted at 2026-08-12 19:04:00 on dist-test-slave-49h3"
server_key: "ec84227b56903a8dc85c87143b6f834d"
server_key_iv: "b4f58318d9b09038413ff483150b002b"
server_key_version: "encryptionkey@0"
I20260812 19:04:00.814309 31963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "2e2ee88466f64b3fb1b7f8e546a4036b"
format_stamp: "Formatted at 2026-08-12 19:04:00 on dist-test-slave-49h3"
server_key: "ec84227b56903a8dc85c87143b6f834d"
server_key_iv: "b4f58318d9b09038413ff483150b002b"
server_key_version: "encryptionkey@0"
I20260812 19:04:00.818591 31963 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 19:04:00.821491 31982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:00.823019 31963 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:04:00.823179 31963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "2e2ee88466f64b3fb1b7f8e546a4036b"
format_stamp: "Formatted at 2026-08-12 19:04:00 on dist-test-slave-49h3"
server_key: "ec84227b56903a8dc85c87143b6f834d"
server_key_iv: "b4f58318d9b09038413ff483150b002b"
server_key_version: "encryptionkey@0"
I20260812 19:04:00.823334 31963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:00.828538 31963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:00.829275 31963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:00.829530 31963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:00.836958 31963 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:39193
I20260812 19:04:00.836984 32052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:39193 every 8 connection(s)
I20260812 19:04:00.838186 31963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:04:00.841508 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 31963
I20260812 19:04:00.841686 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:04:00.842049 26120 external_mini_cluster.cc:1428] Setting key c6ae08517cba10a7e276ad3e1145a967
I20260812 19:04:00.842281 32053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:00.848323 32058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:00.843676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34192 (local address 127.25.130.62:39193)
0812 19:04:00.844399 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:00.844417 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:00.844457 (+ 40us) server_negotiation.cc:407] Connection header received
0812 19:04:00.845245 (+ 788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:00.845266 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:00.845736 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:00.846104 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:00.847010 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:00.847015 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:00.847029 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:04:00.847039 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:00.847102 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:04:00.847119 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:00.847242 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:00.847351 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 19:04:00.847568 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":146,"threads_started":1}
I20260812 19:04:00.849968 32053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e2ee88466f64b3fb1b7f8e546a4036b: Bootstrap starting.
I20260812 19:04:00.852566 32053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e2ee88466f64b3fb1b7f8e546a4036b: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:00.853528 32053 log.cc:824] T 00000000000000000000000000000000 P 2e2ee88466f64b3fb1b7f8e546a4036b: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:00.855971 32053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e2ee88466f64b3fb1b7f8e546a4036b: No bootstrap required, opened a new log
I20260812 19:04:00.860034 32053 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e2ee88466f64b3fb1b7f8e546a4036b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2ee88466f64b3fb1b7f8e546a4036b" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39193 } }
I20260812 19:04:00.860378 32053 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e2ee88466f64b3fb1b7f8e546a4036b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:00.860457 32053 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e2ee88466f64b3fb1b7f8e546a4036b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e2ee88466f64b3fb1b7f8e546a4036b, State: Initialized, Role: FOLLOWER
I20260812 19:04:00.860925 32053 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e2ee88466f64b3fb1b7f8e546a4036b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2ee88466f64b3fb1b7f8e546a4036b" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39193 } }
I20260812 19:04:00.861073 32053 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e2ee88466f64b3fb1b7f8e546a4036b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:04:00.861161 32053 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e2ee88466f64b3fb1b7f8e546a4036b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:04:00.861272 32053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e2ee88466f64b3fb1b7f8e546a4036b [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:00.862552 32053 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e2ee88466f64b3fb1b7f8e546a4036b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2ee88466f64b3fb1b7f8e546a4036b" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39193 } }
I20260812 19:04:00.862931 32053 leader_election.cc:302] T 00000000000000000000000000000000 P 2e2ee88466f64b3fb1b7f8e546a4036b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e2ee88466f64b3fb1b7f8e546a4036b; no voters:
I20260812 19:04:00.863286 32053 leader_election.cc:288] T 00000000000000000000000000000000 P 2e2ee88466f64b3fb1b7f8e546a4036b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:04:00.863413 32060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e2ee88466f64b3fb1b7f8e546a4036b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:04:00.863626 32060 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e2ee88466f64b3fb1b7f8e546a4036b [term 1 LEADER]: Becoming Leader. State: Replica: 2e2ee88466f64b3fb1b7f8e546a4036b, State: Running, Role: LEADER
I20260812 19:04:00.863952 32060 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e2ee88466f64b3fb1b7f8e546a4036b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2ee88466f64b3fb1b7f8e546a4036b" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39193 } }
I20260812 19:04:00.864590 32053 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e2ee88466f64b3fb1b7f8e546a4036b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:04:00.865628 32062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e2ee88466f64b3fb1b7f8e546a4036b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e2ee88466f64b3fb1b7f8e546a4036b. Latest consensus state: current_term: 1 leader_uuid: "2e2ee88466f64b3fb1b7f8e546a4036b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2ee88466f64b3fb1b7f8e546a4036b" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39193 } } }
I20260812 19:04:00.865792 32062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e2ee88466f64b3fb1b7f8e546a4036b [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:00.866091 32061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e2ee88466f64b3fb1b7f8e546a4036b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e2ee88466f64b3fb1b7f8e546a4036b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2ee88466f64b3fb1b7f8e546a4036b" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39193 } } }
I20260812 19:04:00.866179 32061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e2ee88466f64b3fb1b7f8e546a4036b [sys.catalog]: This master's current role is: LEADER
W20260812 19:04:00.870044 32078 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2e2ee88466f64b3fb1b7f8e546a4036b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:04:00.870146 32078 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:04:00.870239 32071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:04:00.871018 32071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:04:00.877568 32071 catalog_manager.cc:1347] Generated new cluster ID: e11927c7d5db445db1bc30ab8f40b150
I20260812 19:04:00.877668 32071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:04:00.891264 32071 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:04:00.891702 32071 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:04:00.900640 32071 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e2ee88466f64b3fb1b7f8e546a4036b: Generated new TSK 0
I20260812 19:04:00.901825 32071 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:04:00.908900 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:0
--local_ip_for_outbound_sockets=127.25.130.1
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:39193
--builtin_ntp_servers=127.25.130.20:37319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 19:04:01.031744 32084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:01.032070 32084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:01.032115 32084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:01.036430 32084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:01.036509 32084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:01.036571 32084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:04:01.041703 32084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:37319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:39193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:01.042968 32084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:01.044524 32084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:01.054648 32096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:01.054673 32092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:01.054766 32090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:01.054756 32084 server_base.cc:1034] running on GCE node
I20260812 19:04:01.055354 32084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:01.056044 32084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:01.057233 32084 hybrid_clock.cc:648] HybridClock initialized: now 1786561441057214 us; error 36 us; skew 500 ppm
I20260812 19:04:01.059291 32084 webserver.cc:460] Webserver started at http://127.25.130.1:39537/ using document root <none> and password file <none>
I20260812 19:04:01.059911 32084 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:01.059988 32084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:01.060205 32084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:01.062465 32084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-0/data/instance:
uuid: "7ba383599ae84f26aab89caf96969796"
format_stamp: "Formatted at 2026-08-12 19:04:01 on dist-test-slave-49h3"
server_key: "63fa65cfe1f638a514009dde8032758f"
server_key_iv: "a034ca67d9e20436b11a73a70c14e2a0"
server_key_version: "encryptionkey@0"
I20260812 19:04:01.063050 32084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance:
uuid: "7ba383599ae84f26aab89caf96969796"
format_stamp: "Formatted at 2026-08-12 19:04:01 on dist-test-slave-49h3"
server_key: "63fa65cfe1f638a514009dde8032758f"
server_key_iv: "a034ca67d9e20436b11a73a70c14e2a0"
server_key_version: "encryptionkey@0"
I20260812 19:04:01.067325 32084 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:04:01.070159 32104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:01.071517 32084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:04:01.071652 32084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "7ba383599ae84f26aab89caf96969796"
format_stamp: "Formatted at 2026-08-12 19:04:01 on dist-test-slave-49h3"
server_key: "63fa65cfe1f638a514009dde8032758f"
server_key_iv: "a034ca67d9e20436b11a73a70c14e2a0"
server_key_version: "encryptionkey@0"
I20260812 19:04:01.071776 32084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:01.076714 32084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:01.077484 32084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:01.077682 32084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:01.078310 32084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:01.079341 32084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:01.079392 32084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:01.079437 32084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:01.079465 32084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:01.088801 32084 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:32911
I20260812 19:04:01.088904 32244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:32911 every 8 connection(s)
I20260812 19:04:01.089802 32084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:04:01.096477 32058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:01.091787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:54613 (local address 127.25.130.62:39193)
0812 19:04:01.091950 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:01.091953 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:01.093115 (+ 1162us) server_negotiation.cc:407] Connection header received
0812 19:04:01.094654 (+ 1539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:01.094658 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:01.094712 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:01.094790 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:01.095974 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:01.095978 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:01.095981 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:04:01.095995 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:01.096017 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:04:01.096025 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:01.096142 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:01.096251 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 19:04:01.096305 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 19:04:01.097345 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 32084
I20260812 19:04:01.097491 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance
I20260812 19:04:01.097497 32248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:01.092180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:39193 (local address 127.25.130.1:54613)
0812 19:04:01.092916 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:01.092962 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:01.094401 (+ 1439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:01.094988 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:01.094995 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:01.095099 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:01.095681 (+ 582us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:04:01.095683 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:01.095729 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:04:01.095808 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:01.096101 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:01.096113 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:01.096119 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 19:04:01.096323 (+ 204us) client_negotiation.cc:240] Negotiation successful
0812 19:04:01.096617 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":117,"threads_started":1}
I20260812 19:04:01.097798 26120 external_mini_cluster.cc:1428] Setting key 49d04fe5cbdc128f3e2ab7f4aa185fa5
I20260812 19:04:01.098908 32246 heartbeater.cc:347] Connected to a master server at 127.25.130.62:39193
I20260812 19:04:01.099195 32246 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:01.099658 32246 heartbeater.cc:511] Master 127.25.130.62:39193 requested a full tablet report, sending...
I20260812 19:04:01.099823 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:0
--local_ip_for_outbound_sockets=127.25.130.2
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:39193
--builtin_ntp_servers=127.25.130.20:37319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 19:04:01.101316 32004 ts_manager.cc:194] Registered new tserver with Master: 7ba383599ae84f26aab89caf96969796 (127.25.130.1:32911)
W20260812 19:04:01.219501 32249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:01.219815 32249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:01.219857 32249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:01.224212 32249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:01.224304 32249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:01.224365 32249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:04:01.229719 32249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:37319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:39193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:01.230885 32249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:01.232317 32249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:04:01.243012 32249 server_base.cc:1034] running on GCE node
W20260812 19:04:01.242954 32255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:01.242957 32256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:01.243022 32258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:01.243690 32249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:01.244488 32249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:01.245662 32249 hybrid_clock.cc:648] HybridClock initialized: now 1786561441245642 us; error 33 us; skew 500 ppm
I20260812 19:04:01.247778 32249 webserver.cc:460] Webserver started at http://127.25.130.2:43219/ using document root <none> and password file <none>
I20260812 19:04:01.248412 32249 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:01.248482 32249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:01.248714 32249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:01.251267 32249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-1/data/instance:
uuid: "67b22f8b38e14dad86cde6a383586199"
format_stamp: "Formatted at 2026-08-12 19:04:01 on dist-test-slave-49h3"
server_key: "b22b159adfecc3dcb5cb00c7ff8d75f8"
server_key_iv: "a5f7ad0c0f519543167dfbebdd89b376"
server_key_version: "encryptionkey@0"
I20260812 19:04:01.251884 32249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance:
uuid: "67b22f8b38e14dad86cde6a383586199"
format_stamp: "Formatted at 2026-08-12 19:04:01 on dist-test-slave-49h3"
server_key: "b22b159adfecc3dcb5cb00c7ff8d75f8"
server_key_iv: "a5f7ad0c0f519543167dfbebdd89b376"
server_key_version: "encryptionkey@0"
I20260812 19:04:01.256207 32249 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 19:04:01.259192 32269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:01.260671 32249 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 19:04:01.260825 32249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "67b22f8b38e14dad86cde6a383586199"
format_stamp: "Formatted at 2026-08-12 19:04:01 on dist-test-slave-49h3"
server_key: "b22b159adfecc3dcb5cb00c7ff8d75f8"
server_key_iv: "a5f7ad0c0f519543167dfbebdd89b376"
server_key_version: "encryptionkey@0"
I20260812 19:04:01.260968 32249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:01.266261 32249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:01.266987 32249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:01.267184 32249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:01.267812 32249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:01.268885 32249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:01.268937 32249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:01.268981 32249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:01.269011 32249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:01.278147 32249 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:37005
I20260812 19:04:01.278173 32399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:37005 every 8 connection(s)
I20260812 19:04:01.279184 32249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:04:01.285682 32058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:01.281203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:34999 (local address 127.25.130.62:39193)
0812 19:04:01.281390 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:01.281395 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:01.282669 (+ 1274us) server_negotiation.cc:407] Connection header received
0812 19:04:01.283961 (+ 1292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:01.283964 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:01.284005 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:01.284069 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:01.285163 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:01.285166 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:01.285168 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:04:01.285181 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:01.285206 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:04:01.285214 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:01.285294 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:01.285485 (+ 191us) server_negotiation.cc:299] Negotiation successful
0812 19:04:01.285542 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 19:04:01.286545 32402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:01.281741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:39193 (local address 127.25.130.2:34999)
0812 19:04:01.282503 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:01.282550 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:01.283752 (+ 1202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:01.284258 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:01.284266 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:01.284361 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:01.284897 (+ 536us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:04:01.284899 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:01.284942 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:04:01.285015 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:01.285284 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:01.285290 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:01.285294 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 19:04:01.285430 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 19:04:01.285663 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":130,"threads_started":1}
I20260812 19:04:01.287689 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 32249
I20260812 19:04:01.287810 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance
I20260812 19:04:01.287885 32400 heartbeater.cc:347] Connected to a master server at 127.25.130.62:39193
I20260812 19:04:01.288144 26120 external_mini_cluster.cc:1428] Setting key 98013fb0f5c6e9f69fe12aedd5a75fd2
I20260812 19:04:01.288146 32400 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:01.288600 32400 heartbeater.cc:511] Master 127.25.130.62:39193 requested a full tablet report, sending...
I20260812 19:04:01.289865 32004 ts_manager.cc:194] Registered new tserver with Master: 67b22f8b38e14dad86cde6a383586199 (127.25.130.2:37005)
I20260812 19:04:01.290452 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:0
--local_ip_for_outbound_sockets=127.25.130.3
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:39193
--builtin_ntp_servers=127.25.130.20:37319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 19:04:01.408203 32404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:01.408514 32404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:01.408573 32404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:01.412922 32404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:01.413007 32404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:01.413090 32404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:04:01.418200 32404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:37319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:39193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:01.419410 32404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:01.420895 32404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:01.431282 32411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:01.431551 32404 server_base.cc:1034] running on GCE node
W20260812 19:04:01.431285 32413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:01.431285 32410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:01.432459 32404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:01.433241 32404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:01.434402 32404 hybrid_clock.cc:648] HybridClock initialized: now 1786561441434382 us; error 72 us; skew 500 ppm
I20260812 19:04:01.436411 32404 webserver.cc:460] Webserver started at http://127.25.130.3:36981/ using document root <none> and password file <none>
I20260812 19:04:01.437031 32404 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:01.437114 32404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:01.437332 32404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:01.439684 32404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-2/data/instance:
uuid: "f5cc516f0cef462ab1f95f1ea8e43dc0"
format_stamp: "Formatted at 2026-08-12 19:04:01 on dist-test-slave-49h3"
server_key: "1ee61168a934255fafcaa72dfe484076"
server_key_iv: "a4831b82851c97a5116900bfd0580def"
server_key_version: "encryptionkey@0"
I20260812 19:04:01.440320 32404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance:
uuid: "f5cc516f0cef462ab1f95f1ea8e43dc0"
format_stamp: "Formatted at 2026-08-12 19:04:01 on dist-test-slave-49h3"
server_key: "1ee61168a934255fafcaa72dfe484076"
server_key_iv: "a4831b82851c97a5116900bfd0580def"
server_key_version: "encryptionkey@0"
I20260812 19:04:01.444769 32404 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 19:04:01.447834 32420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:01.449537 32404 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:04:01.449702 32404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "f5cc516f0cef462ab1f95f1ea8e43dc0"
format_stamp: "Formatted at 2026-08-12 19:04:01 on dist-test-slave-49h3"
server_key: "1ee61168a934255fafcaa72dfe484076"
server_key_iv: "a4831b82851c97a5116900bfd0580def"
server_key_version: "encryptionkey@0"
I20260812 19:04:01.449870 32404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:01.457866 32404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:01.458611 32404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:01.458789 32404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:01.459386 32404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:01.460441 32404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:01.460496 32404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:01.460553 32404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:01.460578 32404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:01.471204 32404 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:41501
I20260812 19:04:01.471233 32559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:41501 every 8 connection(s)
I20260812 19:04:01.472261 32404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:04:01.478402 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 32404
I20260812 19:04:01.478542 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance
I20260812 19:04:01.478832 26120 external_mini_cluster.cc:1428] Setting key 34cc3b42831e0f7585e08d07d4626a5c
I20260812 19:04:01.478997 32058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:01.474308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:33887 (local address 127.25.130.62:39193)
0812 19:04:01.474450 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:01.474454 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:01.475633 (+ 1179us) server_negotiation.cc:407] Connection header received
0812 19:04:01.477000 (+ 1367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:01.477004 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:01.477057 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:01.477138 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:01.478465 (+ 1327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:01.478469 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:01.478471 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:04:01.478487 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:01.478517 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:04:01.478527 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:01.478653 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:01.478795 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 19:04:01.478848 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 19:04:01.480158 32562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:01.474695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:39193 (local address 127.25.130.3:33887)
0812 19:04:01.475432 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:01.475479 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:01.476750 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:01.477350 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:01.477358 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:01.477497 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:01.478086 (+ 589us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:04:01.478089 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:01.478151 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:04:01.478280 (+ 129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:01.478639 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:01.478651 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:01.478657 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 19:04:01.478837 (+ 180us) client_negotiation.cc:240] Negotiation successful
0812 19:04:01.479117 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":123,"threads_started":1}
I20260812 19:04:01.481314 32560 heartbeater.cc:347] Connected to a master server at 127.25.130.62:39193
I20260812 19:04:01.481598 32560 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:01.482043 32560 heartbeater.cc:511] Master 127.25.130.62:39193 requested a full tablet report, sending...
I20260812 19:04:01.483155 32003 ts_manager.cc:194] Registered new tserver with Master: f5cc516f0cef462ab1f95f1ea8e43dc0 (127.25.130.3:41501)
I20260812 19:04:01.493363 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:04:01.496990 32058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:01.495519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34206 (local address 127.25.130.62:39193)
0812 19:04:01.495659 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:01.495662 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:01.495723 (+ 61us) server_negotiation.cc:407] Connection header received
0812 19:04:01.495850 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:01.495853 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:01.495888 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:01.495966 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:01.496664 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:01.496667 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:01.496669 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:04:01.496681 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:01.496702 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:04:01.496709 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:01.496789 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:01.496838 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 19:04:01.496875 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 19:04:01.500069 32003 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34206:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:04:01.503389 32003 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:04:01.515599 32578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:01.512888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:41501 (local address 127.0.0.1:57542)
0812 19:04:01.513403 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:01.513416 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:01.513515 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:01.514456 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:01.514459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:01.514476 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:01.514851 (+ 375us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:04:01.514853 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:01.514867 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:04:01.514899 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:01.515268 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:01.515271 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:01.515275 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 19:04:01.515358 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 19:04:01.515432 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":101,"threads_started":1}
I20260812 19:04:01.515910 32576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:01.512185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43802 (local address 127.25.130.2:37005)
0812 19:04:01.512853 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:01.512866 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:01.513119 (+ 253us) server_negotiation.cc:407] Connection header received
0812 19:04:01.513341 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:01.513348 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:01.513598 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:01.513734 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:01.515033 (+ 1299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:01.515039 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:01.515048 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:04:01.515059 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:01.515098 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:04:01.515124 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:01.515226 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:01.515661 (+ 435us) server_negotiation.cc:299] Negotiation successful
0812 19:04:01.515756 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":146,"threads_started":1}
I20260812 19:04:01.516238 32577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:01.512390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46058 (local address 127.25.130.1:32911)
0812 19:04:01.513044 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:01.513057 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:01.513742 (+ 685us) server_negotiation.cc:407] Connection header received
0812 19:04:01.513804 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:01.513809 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:01.513944 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:01.514072 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:01.515469 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:01.515474 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:01.515485 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:04:01.515498 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:01.515539 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:04:01.515580 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:01.515661 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:01.515847 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 19:04:01.515921 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":114,"threads_started":1}
I20260812 19:04:01.516326 32579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:01.513326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57542 (local address 127.25.130.3:41501)
0812 19:04:01.513688 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:01.513700 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:01.513725 (+ 25us) server_negotiation.cc:407] Connection header received
0812 19:04:01.513777 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:01.513782 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:01.513944 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:01.514072 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:01.515065 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:01.515072 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:01.515084 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:04:01.515096 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:01.515148 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:04:01.515174 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:01.515270 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:01.516098 (+ 828us) server_negotiation.cc:299] Negotiation successful
0812 19:04:01.516184 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":168,"threads_started":1}
I20260812 19:04:01.515599 32573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:01.512051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:37005 (local address 127.0.0.1:43802)
0812 19:04:01.512973 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:01.513027 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:01.513203 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:01.513879 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:01.513884 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:01.513961 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:01.514561 (+ 600us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:04:01.514563 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:01.514585 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:04:01.514899 (+ 314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:01.515220 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:01.515230 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:01.515236 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 19:04:01.515354 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 19:04:01.515427 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":755,"thread_start_us":99,"threads_started":1}
I20260812 19:04:01.516695 32575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:01.512123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:32911 (local address 127.0.0.1:46058)
0812 19:04:01.513179 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:01.513191 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:01.513732 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:01.514687 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:01.514690 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:01.514717 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:01.515299 (+ 582us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:04:01.515301 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:01.515311 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:04:01.515336 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:01.515733 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:01.515735 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:01.515737 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 19:04:01.516522 (+ 785us) client_negotiation.cc:240] Negotiation successful
0812 19:04:01.516576 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":997,"thread_start_us":88,"threads_started":1}
I20260812 19:04:01.519670 32480 tablet_service.cc:1467] Processing CreateTablet for tablet 84441022953d48e8b532661b026cd986 (DEFAULT_TABLE table=test-table [id=8f83e97456b64f67b4c5e0f40d976d8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:04:01.519670 32321 tablet_service.cc:1467] Processing CreateTablet for tablet 84441022953d48e8b532661b026cd986 (DEFAULT_TABLE table=test-table [id=8f83e97456b64f67b4c5e0f40d976d8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:04:01.520951 32480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84441022953d48e8b532661b026cd986. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:01.520951 32321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84441022953d48e8b532661b026cd986. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:01.523684 32163 tablet_service.cc:1467] Processing CreateTablet for tablet 84441022953d48e8b532661b026cd986 (DEFAULT_TABLE table=test-table [id=8f83e97456b64f67b4c5e0f40d976d8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:04:01.524845 32163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84441022953d48e8b532661b026cd986. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:01.529480 32580 tablet_bootstrap.cc:492] T 84441022953d48e8b532661b026cd986 P f5cc516f0cef462ab1f95f1ea8e43dc0: Bootstrap starting.
I20260812 19:04:01.530728 32581 tablet_bootstrap.cc:492] T 84441022953d48e8b532661b026cd986 P 67b22f8b38e14dad86cde6a383586199: Bootstrap starting.
I20260812 19:04:01.532262 32580 tablet_bootstrap.cc:654] T 84441022953d48e8b532661b026cd986 P f5cc516f0cef462ab1f95f1ea8e43dc0: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:01.532236 32583 tablet_bootstrap.cc:492] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796: Bootstrap starting.
I20260812 19:04:01.532755 32581 tablet_bootstrap.cc:654] T 84441022953d48e8b532661b026cd986 P 67b22f8b38e14dad86cde6a383586199: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:01.533181 32580 log.cc:824] T 84441022953d48e8b532661b026cd986 P f5cc516f0cef462ab1f95f1ea8e43dc0: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:01.533664 32581 log.cc:824] T 84441022953d48e8b532661b026cd986 P 67b22f8b38e14dad86cde6a383586199: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:01.535176 32583 tablet_bootstrap.cc:654] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:01.535444 32580 tablet_bootstrap.cc:492] T 84441022953d48e8b532661b026cd986 P f5cc516f0cef462ab1f95f1ea8e43dc0: No bootstrap required, opened a new log
I20260812 19:04:01.535651 32580 ts_tablet_manager.cc:1397] T 84441022953d48e8b532661b026cd986 P f5cc516f0cef462ab1f95f1ea8e43dc0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 19:04:01.536116 32583 log.cc:824] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:01.536224 32581 tablet_bootstrap.cc:492] T 84441022953d48e8b532661b026cd986 P 67b22f8b38e14dad86cde6a383586199: No bootstrap required, opened a new log
I20260812 19:04:01.536417 32581 ts_tablet_manager.cc:1397] T 84441022953d48e8b532661b026cd986 P 67b22f8b38e14dad86cde6a383586199: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 19:04:01.538534 32583 tablet_bootstrap.cc:492] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796: No bootstrap required, opened a new log
I20260812 19:04:01.538774 32583 ts_tablet_manager.cc:1397] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260812 19:04:01.539940 32581 raft_consensus.cc:348] T 84441022953d48e8b532661b026cd986 P 67b22f8b38e14dad86cde6a383586199 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67b22f8b38e14dad86cde6a383586199" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37005 } } peers { permanent_uuid: "7ba383599ae84f26aab89caf96969796" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32911 } } peers { permanent_uuid: "f5cc516f0cef462ab1f95f1ea8e43dc0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 41501 } }
I20260812 19:04:01.539964 32580 raft_consensus.cc:348] T 84441022953d48e8b532661b026cd986 P f5cc516f0cef462ab1f95f1ea8e43dc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67b22f8b38e14dad86cde6a383586199" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37005 } } peers { permanent_uuid: "7ba383599ae84f26aab89caf96969796" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32911 } } peers { permanent_uuid: "f5cc516f0cef462ab1f95f1ea8e43dc0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 41501 } }
I20260812 19:04:01.540205 32581 raft_consensus.cc:374] T 84441022953d48e8b532661b026cd986 P 67b22f8b38e14dad86cde6a383586199 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:01.540223 32580 raft_consensus.cc:374] T 84441022953d48e8b532661b026cd986 P f5cc516f0cef462ab1f95f1ea8e43dc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:01.540266 32581 raft_consensus.cc:729] T 84441022953d48e8b532661b026cd986 P 67b22f8b38e14dad86cde6a383586199 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67b22f8b38e14dad86cde6a383586199, State: Initialized, Role: FOLLOWER
I20260812 19:04:01.540268 32580 raft_consensus.cc:729] T 84441022953d48e8b532661b026cd986 P f5cc516f0cef462ab1f95f1ea8e43dc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5cc516f0cef462ab1f95f1ea8e43dc0, State: Initialized, Role: FOLLOWER
I20260812 19:04:01.540753 32581 consensus_queue.cc:260] T 84441022953d48e8b532661b026cd986 P 67b22f8b38e14dad86cde6a383586199 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67b22f8b38e14dad86cde6a383586199" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37005 } } peers { permanent_uuid: "7ba383599ae84f26aab89caf96969796" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32911 } } peers { permanent_uuid: "f5cc516f0cef462ab1f95f1ea8e43dc0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 41501 } }
I20260812 19:04:01.540758 32580 consensus_queue.cc:260] T 84441022953d48e8b532661b026cd986 P f5cc516f0cef462ab1f95f1ea8e43dc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67b22f8b38e14dad86cde6a383586199" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37005 } } peers { permanent_uuid: "7ba383599ae84f26aab89caf96969796" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32911 } } peers { permanent_uuid: "f5cc516f0cef462ab1f95f1ea8e43dc0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 41501 } }
I20260812 19:04:01.541824 32400 heartbeater.cc:503] Master 127.25.130.62:39193 was elected leader, sending a full tablet report...
I20260812 19:04:01.541949 32581 ts_tablet_manager.cc:1428] T 84441022953d48e8b532661b026cd986 P 67b22f8b38e14dad86cde6a383586199: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260812 19:04:01.541951 32580 ts_tablet_manager.cc:1428] T 84441022953d48e8b532661b026cd986 P f5cc516f0cef462ab1f95f1ea8e43dc0: Time spent starting tablet: real 0.006s user 0.004s sys 0.004s
I20260812 19:04:01.542397 32560 heartbeater.cc:503] Master 127.25.130.62:39193 was elected leader, sending a full tablet report...
I20260812 19:04:01.544575 32583 raft_consensus.cc:348] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67b22f8b38e14dad86cde6a383586199" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37005 } } peers { permanent_uuid: "7ba383599ae84f26aab89caf96969796" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32911 } } peers { permanent_uuid: "f5cc516f0cef462ab1f95f1ea8e43dc0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 41501 } }
I20260812 19:04:01.544922 32583 raft_consensus.cc:374] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:01.544987 32583 raft_consensus.cc:729] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ba383599ae84f26aab89caf96969796, State: Initialized, Role: FOLLOWER
I20260812 19:04:01.545570 32583 consensus_queue.cc:260] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67b22f8b38e14dad86cde6a383586199" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37005 } } peers { permanent_uuid: "7ba383599ae84f26aab89caf96969796" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32911 } } peers { permanent_uuid: "f5cc516f0cef462ab1f95f1ea8e43dc0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 41501 } }
I20260812 19:04:01.546337 32246 heartbeater.cc:503] Master 127.25.130.62:39193 was elected leader, sending a full tablet report...
I20260812 19:04:01.546808 32583 ts_tablet_manager.cc:1428] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
W20260812 19:04:01.591064 32247 tablet.cc:2369] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:04:01.670032 32591 raft_consensus.cc:482] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:04:01.670284 32591 raft_consensus.cc:504] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67b22f8b38e14dad86cde6a383586199" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37005 } } peers { permanent_uuid: "7ba383599ae84f26aab89caf96969796" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32911 } } peers { permanent_uuid: "f5cc516f0cef462ab1f95f1ea8e43dc0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 41501 } }
I20260812 19:04:01.671517 32591 leader_election.cc:288] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67b22f8b38e14dad86cde6a383586199 (127.25.130.2:37005), f5cc516f0cef462ab1f95f1ea8e43dc0 (127.25.130.3:41501)
I20260812 19:04:01.673805 32576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:01.671947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:42661 (local address 127.25.130.2:37005)
0812 19:04:01.672187 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:01.672192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:01.672336 (+ 144us) server_negotiation.cc:407] Connection header received
0812 19:04:01.672407 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:01.672410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:01.672451 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:01.672540 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:01.673374 (+ 834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:01.673377 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:01.673380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:04:01.673391 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:01.673412 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:04:01.673421 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:01.673534 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:01.673631 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 19:04:01.673668 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260812 19:04:01.673805 32579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:01.672040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:51651 (local address 127.25.130.3:41501)
0812 19:04:01.672212 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:01.672216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:01.672282 (+ 66us) server_negotiation.cc:407] Connection header received
0812 19:04:01.672407 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:01.672410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:01.672451 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:01.672540 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:01.673354 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:01.673358 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:01.673360 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:04:01.673374 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:01.673399 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:04:01.673410 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:01.673536 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:01.673614 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 19:04:01.673661 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260812 19:04:01.673847 32594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:01.671829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:41501 (local address 127.25.130.1:51651)
0812 19:04:01.672171 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:01.672188 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:01.672313 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:01.672575 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:01.672578 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:01.672595 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:01.673150 (+ 555us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:04:01.673152 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:01.673165 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:04:01.673203 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:01.673525 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:01.673528 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:01.673531 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 19:04:01.673617 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 19:04:01.673674 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":81,"threads_started":1}
I20260812 19:04:01.673847 32593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:01.671829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:37005 (local address 127.25.130.1:42661)
0812 19:04:01.672253 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:01.672265 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:01.672330 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:01.672575 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:01.672578 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:01.672595 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:01.673219 (+ 624us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:04:01.673221 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:01.673230 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:04:01.673249 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:01.673528 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:01.673530 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:01.673533 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 19:04:01.673617 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 19:04:01.673684 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":64,"thread_start_us":81,"threads_started":1}
I20260812 19:04:01.674678 32506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84441022953d48e8b532661b026cd986" candidate_uuid: "7ba383599ae84f26aab89caf96969796" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5cc516f0cef462ab1f95f1ea8e43dc0" is_pre_election: true
I20260812 19:04:01.674844 32346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84441022953d48e8b532661b026cd986" candidate_uuid: "7ba383599ae84f26aab89caf96969796" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67b22f8b38e14dad86cde6a383586199" is_pre_election: true
I20260812 19:04:01.675016 32506 raft_consensus.cc:2454] T 84441022953d48e8b532661b026cd986 P f5cc516f0cef462ab1f95f1ea8e43dc0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ba383599ae84f26aab89caf96969796 in term 0.
I20260812 19:04:01.675184 32346 raft_consensus.cc:2454] T 84441022953d48e8b532661b026cd986 P 67b22f8b38e14dad86cde6a383586199 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ba383599ae84f26aab89caf96969796 in term 0.
I20260812 19:04:01.675527 32107 leader_election.cc:302] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ba383599ae84f26aab89caf96969796, f5cc516f0cef462ab1f95f1ea8e43dc0; no voters:
I20260812 19:04:01.675850 32591 raft_consensus.cc:2781] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:04:01.675940 32591 raft_consensus.cc:482] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:04:01.675992 32591 raft_consensus.cc:3037] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:01.677109 32591 raft_consensus.cc:504] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67b22f8b38e14dad86cde6a383586199" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37005 } } peers { permanent_uuid: "7ba383599ae84f26aab89caf96969796" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32911 } } peers { permanent_uuid: "f5cc516f0cef462ab1f95f1ea8e43dc0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 41501 } }
I20260812 19:04:01.677521 32591 leader_election.cc:288] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796 [CANDIDATE]: Term 1 election: Requested vote from peers 67b22f8b38e14dad86cde6a383586199 (127.25.130.2:37005), f5cc516f0cef462ab1f95f1ea8e43dc0 (127.25.130.3:41501)
I20260812 19:04:01.677870 32346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84441022953d48e8b532661b026cd986" candidate_uuid: "7ba383599ae84f26aab89caf96969796" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67b22f8b38e14dad86cde6a383586199"
I20260812 19:04:01.677897 32506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84441022953d48e8b532661b026cd986" candidate_uuid: "7ba383599ae84f26aab89caf96969796" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5cc516f0cef462ab1f95f1ea8e43dc0"
I20260812 19:04:01.677999 32346 raft_consensus.cc:3037] T 84441022953d48e8b532661b026cd986 P 67b22f8b38e14dad86cde6a383586199 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:01.678011 32506 raft_consensus.cc:3037] T 84441022953d48e8b532661b026cd986 P f5cc516f0cef462ab1f95f1ea8e43dc0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:01.679057 32346 raft_consensus.cc:2454] T 84441022953d48e8b532661b026cd986 P 67b22f8b38e14dad86cde6a383586199 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ba383599ae84f26aab89caf96969796 in term 1.
I20260812 19:04:01.679067 32506 raft_consensus.cc:2454] T 84441022953d48e8b532661b026cd986 P f5cc516f0cef462ab1f95f1ea8e43dc0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ba383599ae84f26aab89caf96969796 in term 1.
I20260812 19:04:01.679359 32112 leader_election.cc:302] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67b22f8b38e14dad86cde6a383586199, 7ba383599ae84f26aab89caf96969796; no voters:
I20260812 19:04:01.679550 32591 raft_consensus.cc:2781] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:04:01.679822 32591 raft_consensus.cc:686] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796 [term 1 LEADER]: Becoming Leader. State: Replica: 7ba383599ae84f26aab89caf96969796, State: Running, Role: LEADER
I20260812 19:04:01.680152 32591 consensus_queue.cc:237] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67b22f8b38e14dad86cde6a383586199" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37005 } } peers { permanent_uuid: "7ba383599ae84f26aab89caf96969796" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32911 } } peers { permanent_uuid: "f5cc516f0cef462ab1f95f1ea8e43dc0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 41501 } }
I20260812 19:04:01.683490 32004 catalog_manager.cc:5495] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ba383599ae84f26aab89caf96969796 (127.25.130.1). New cstate: current_term: 1 leader_uuid: "7ba383599ae84f26aab89caf96969796" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67b22f8b38e14dad86cde6a383586199" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ba383599ae84f26aab89caf96969796" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5cc516f0cef462ab1f95f1ea8e43dc0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 41501 } health_report { overall_health: UNKNOWN } } }
I20260812 19:04:01.708087 32577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:01.706544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46068 (local address 127.25.130.1:32911)
0812 19:04:01.706706 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:01.706711 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:01.706725 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:04:01.706817 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:01.706819 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:01.706883 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:01.706956 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:01.707657 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:01.707661 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:01.707663 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:04:01.707674 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:01.707693 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:04:01.707702 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:01.707778 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:01.707873 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 19:04:01.707915 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 19:04:01.715140 32346 raft_consensus.cc:1261] T 84441022953d48e8b532661b026cd986 P 67b22f8b38e14dad86cde6a383586199 [term 1 FOLLOWER]: Refusing update from remote peer 7ba383599ae84f26aab89caf96969796: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:04:01.715144 32506 raft_consensus.cc:1261] T 84441022953d48e8b532661b026cd986 P f5cc516f0cef462ab1f95f1ea8e43dc0 [term 1 FOLLOWER]: Refusing update from remote peer 7ba383599ae84f26aab89caf96969796: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:04:01.715924 32595 consensus_queue.cc:1035] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5cc516f0cef462ab1f95f1ea8e43dc0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 41501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:04:01.716141 32591 consensus_queue.cc:1035] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67b22f8b38e14dad86cde6a383586199" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 19:04:01.723220 32561 tablet.cc:2369] T 84441022953d48e8b532661b026cd986 P f5cc516f0cef462ab1f95f1ea8e43dc0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:04:01.724897 32599 mvcc.cc:205] Tried to move back new op lower bound from 7317755665255739392 to 7317755665124589568. Current Snapshot: MvccSnapshot[applied={T|T < 7317755665255739392}]
I20260812 19:04:01.725114 32598 mvcc.cc:205] Tried to move back new op lower bound from 7317755665255739392 to 7317755665124589568. Current Snapshot: MvccSnapshot[applied={T|T < 7317755665255739392}]
I20260812 19:04:01.725898 32600 mvcc.cc:205] Tried to move back new op lower bound from 7317755665255739392 to 7317755665124589568. Current Snapshot: MvccSnapshot[applied={T|T < 7317755665255739392}]
I20260812 19:04:01.736605 32003 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:04:01.736814 32003 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:04:01.739557 32003 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2e2ee88466f64b3fb1b7f8e546a4036b: Sending DeleteTablet for 3 replicas of tablet 84441022953d48e8b532661b026cd986
I20260812 19:04:01.740535 32321 tablet_service.cc:1514] Processing DeleteTablet for tablet 84441022953d48e8b532661b026cd986 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:04:01 UTC) from {username='slave'} at 127.0.0.1:43802
I20260812 19:04:01.740566 32163 tablet_service.cc:1514] Processing DeleteTablet for tablet 84441022953d48e8b532661b026cd986 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:04:01 UTC) from {username='slave'} at 127.0.0.1:46058
I20260812 19:04:01.740710 32480 tablet_service.cc:1514] Processing DeleteTablet for tablet 84441022953d48e8b532661b026cd986 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:04:01 UTC) from {username='slave'} at 127.0.0.1:57542
I20260812 19:04:01.741186 32609 tablet_replica.cc:330] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796: stopping tablet replica
I20260812 19:04:01.741639 32609 raft_consensus.cc:2229] T 84441022953d48e8b532661b026cd986 P 7ba383599ae84f26aab89caf96969796 [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:04:01.741681 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 32084
I20260812 19:04:01.746902 32610 tablet_replica.cc:330] T 84441022953d48e8b532661b026cd986 P f5cc516f0cef462ab1f95f1ea8e43dc0: stopping tablet replica
I20260812 19:04:01.747296 32610 raft_consensus.cc:2229] T 84441022953d48e8b532661b026cd986 P f5cc516f0cef462ab1f95f1ea8e43dc0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:04:01.747665 32610 raft_consensus.cc:2258] T 84441022953d48e8b532661b026cd986 P f5cc516f0cef462ab1f95f1ea8e43dc0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:04:01.749307 32610 ts_tablet_manager.cc:1905] T 84441022953d48e8b532661b026cd986 P f5cc516f0cef462ab1f95f1ea8e43dc0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:04:01.751663 32610 ts_tablet_manager.cc:1918] T 84441022953d48e8b532661b026cd986 P f5cc516f0cef462ab1f95f1ea8e43dc0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:04:01.751776 32610 log.cc:1196] T 84441022953d48e8b532661b026cd986 P f5cc516f0cef462ab1f95f1ea8e43dc0: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-2/wal/wals/84441022953d48e8b532661b026cd986
I20260812 19:04:01.752103 32610 ts_tablet_manager.cc:1939] T 84441022953d48e8b532661b026cd986 P f5cc516f0cef462ab1f95f1ea8e43dc0: Deleting consensus metadata
I20260812 19:04:01.752274 32607 tablet_replica.cc:330] T 84441022953d48e8b532661b026cd986 P 67b22f8b38e14dad86cde6a383586199: stopping tablet replica
I20260812 19:04:01.752699 32607 raft_consensus.cc:2229] T 84441022953d48e8b532661b026cd986 P 67b22f8b38e14dad86cde6a383586199 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:04:01.753110 32607 raft_consensus.cc:2258] T 84441022953d48e8b532661b026cd986 P 67b22f8b38e14dad86cde6a383586199 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:04:01.753634 31985 catalog_manager.cc:4841] TS f5cc516f0cef462ab1f95f1ea8e43dc0 (127.25.130.3:41501): tablet 84441022953d48e8b532661b026cd986 (table test-table [id=8f83e97456b64f67b4c5e0f40d976d8d]) successfully deleted
I20260812 19:04:01.758890 32607 ts_tablet_manager.cc:1905] T 84441022953d48e8b532661b026cd986 P 67b22f8b38e14dad86cde6a383586199: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:04:01.761284 32607 ts_tablet_manager.cc:1918] T 84441022953d48e8b532661b026cd986 P 67b22f8b38e14dad86cde6a383586199: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:04:01.761396 32607 log.cc:1196] T 84441022953d48e8b532661b026cd986 P 67b22f8b38e14dad86cde6a383586199: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561415988404-26120-0/minicluster-data/ts-1/wal/wals/84441022953d48e8b532661b026cd986
I20260812 19:04:01.761775 32607 ts_tablet_manager.cc:1939] T 84441022953d48e8b532661b026cd986 P 67b22f8b38e14dad86cde6a383586199: Deleting consensus metadata
I20260812 19:04:01.762737 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 32249
W20260812 19:04:01.762813 31987 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.130.1:32911 (error 108)
I20260812 19:04:01.767930 32575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:01.767608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:32911 (local address 127.0.0.1:46076)
0812 19:04:01.767760 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:01.767838 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.1:32911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260812 19:04:01.768136 31987 catalog_manager.cc:4584] TS 7ba383599ae84f26aab89caf96969796 (127.25.130.1:32911): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 84441022953d48e8b532661b026cd986: Network error: Client connection negotiation failed: client connection to 127.25.130.1:32911: connect: Connection refused (error 111)
I20260812 19:04:01.771817 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 32404
I20260812 19:04:01.774925 32575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:01.772001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:37005 (local address 127.0.0.1:43806)
0812 19:04:01.774752 (+ 2751us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:01.774824 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.2:37005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2639}
I20260812 19:04:01.779989 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 31963
2026-08-12T19:04:01Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1149 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 19:04:01.805759 26120 mini_oidc.cc:152] Starting JWKS server
I20260812 19:04:01.807570 26120 webserver.cc:460] Webserver started at https://127.0.0.1:36731/ using document root <none> and password file <none>
I20260812 19:04:01.807727 26120 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 19:04:01.807971 26120 webserver.cc:460] Webserver started at http://127.0.0.1:33623/ using document root <none> and password file <none>
2026-08-12T19:04:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:04:01Z Disabled control of system clock
I20260812 19:04:01.818799 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:42523
--webserver_interface=127.25.130.62
--webserver_port=0
--builtin_ntp_servers=127.25.130.20:38773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:42523
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36731/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561415988404-26120-0/testchainca.cert with env {}
W20260812 19:04:01.934680 32620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:01.935098 32620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:01.935171 32620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:01.941531 32620 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:04:01.941648 32620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:01.941673 32620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:01.941708 32620 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:04:01.941732 32620 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 19:04:01.941766 32620 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 19:04:01.941783 32620 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36731/jwks.json
I20260812 19:04:01.948522 32620 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:38773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:42523
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:42523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36731/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561415988404-26120-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:01.949802 32620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:01.950874 32620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:01.960006 32629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:01.960023 32628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:01.960263 32620 server_base.cc:1034] running on GCE node
W20260812 19:04:01.960006 32632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:01.960960 32620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:01.962247 32620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:01.963436 32620 hybrid_clock.cc:648] HybridClock initialized: now 1786561441963415 us; error 36 us; skew 500 ppm
I20260812 19:04:01.965436 32620 webserver.cc:460] Webserver started at http://127.25.130.62:43695/ using document root <none> and password file <none>
I20260812 19:04:01.966104 32620 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:01.966187 32620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:01.966394 32620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:01.968580 32620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "f42d7998d04a4c0caad3310199dec7d5"
format_stamp: "Formatted at 2026-08-12 19:04:01 on dist-test-slave-49h3"
server_key: "cd17c58a23985eb1fee1d82254a6f708"
server_key_iv: "51df6d6979f2af8e92ce0c0a2bb6209b"
server_key_version: "encryptionkey@0"
I20260812 19:04:01.969175 32620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "f42d7998d04a4c0caad3310199dec7d5"
format_stamp: "Formatted at 2026-08-12 19:04:01 on dist-test-slave-49h3"
server_key: "cd17c58a23985eb1fee1d82254a6f708"
server_key_iv: "51df6d6979f2af8e92ce0c0a2bb6209b"
server_key_version: "encryptionkey@0"
I20260812 19:04:01.973049 32620 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260812 19:04:01.975656 32641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:01.976891 32620 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:04:01.977016 32620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "f42d7998d04a4c0caad3310199dec7d5"
format_stamp: "Formatted at 2026-08-12 19:04:01 on dist-test-slave-49h3"
server_key: "cd17c58a23985eb1fee1d82254a6f708"
server_key_iv: "51df6d6979f2af8e92ce0c0a2bb6209b"
server_key_version: "encryptionkey@0"
I20260812 19:04:01.977119 32620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:01.993899 32620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:01.994702 32620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:01.994907 32620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:02.002130 32620 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:42523
I20260812 19:04:02.002187 32704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:42523 every 8 connection(s)
I20260812 19:04:02.003181 32620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:04:02.006160 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 32620
I20260812 19:04:02.006294 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:04:02.006621 26120 external_mini_cluster.cc:1428] Setting key e73defa009b2749bd4cbf2087e8cdd22
I20260812 19:04:02.007198 32705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:02.015090 32705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f42d7998d04a4c0caad3310199dec7d5: Bootstrap starting.
I20260812 19:04:02.017052 32708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:02.008194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58110 (local address 127.25.130.62:42523)
0812 19:04:02.008939 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:02.008957 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:02.008997 (+ 40us) server_negotiation.cc:407] Connection header received
0812 19:04:02.009763 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:02.009783 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:02.010221 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:02.010543 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:02.010982 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:02.012086 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:02.013328 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:02.013934 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:02.014078 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:02.015025 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:02.015030 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:02.015046 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:04:02.015508 (+ 462us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:02.015576 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:04:02.015610 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:02.015688 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:02.015860 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 19:04:02.016157 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":165,"threads_started":1}
I20260812 19:04:02.017941 32705 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f42d7998d04a4c0caad3310199dec7d5: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:02.018772 32705 log.cc:824] T 00000000000000000000000000000000 P f42d7998d04a4c0caad3310199dec7d5: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:02.020234 26120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:04:02.020987 32705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f42d7998d04a4c0caad3310199dec7d5: No bootstrap required, opened a new log
I20260812 19:04:02.024758 32705 raft_consensus.cc:348] T 00000000000000000000000000000000 P f42d7998d04a4c0caad3310199dec7d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f42d7998d04a4c0caad3310199dec7d5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42523 } }
I20260812 19:04:02.025043 32705 raft_consensus.cc:374] T 00000000000000000000000000000000 P f42d7998d04a4c0caad3310199dec7d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:02.025107 32705 raft_consensus.cc:729] T 00000000000000000000000000000000 P f42d7998d04a4c0caad3310199dec7d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f42d7998d04a4c0caad3310199dec7d5, State: Initialized, Role: FOLLOWER
I20260812 19:04:02.025574 32705 consensus_queue.cc:260] T 00000000000000000000000000000000 P f42d7998d04a4c0caad3310199dec7d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f42d7998d04a4c0caad3310199dec7d5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42523 } }
I20260812 19:04:02.025722 32705 raft_consensus.cc:388] T 00000000000000000000000000000000 P f42d7998d04a4c0caad3310199dec7d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:04:02.025789 32705 raft_consensus.cc:482] T 00000000000000000000000000000000 P f42d7998d04a4c0caad3310199dec7d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:04:02.025892 32705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f42d7998d04a4c0caad3310199dec7d5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:02.026921 32705 raft_consensus.cc:504] T 00000000000000000000000000000000 P f42d7998d04a4c0caad3310199dec7d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f42d7998d04a4c0caad3310199dec7d5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42523 } }
I20260812 19:04:02.027279 32705 leader_election.cc:302] T 00000000000000000000000000000000 P f42d7998d04a4c0caad3310199dec7d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f42d7998d04a4c0caad3310199dec7d5; no voters:
I20260812 19:04:02.027660 32705 leader_election.cc:288] T 00000000000000000000000000000000 P f42d7998d04a4c0caad3310199dec7d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:04:02.027791 32710 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f42d7998d04a4c0caad3310199dec7d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:04:02.028034 32710 raft_consensus.cc:686] T 00000000000000000000000000000000 P f42d7998d04a4c0caad3310199dec7d5 [term 1 LEADER]: Becoming Leader. State: Replica: f42d7998d04a4c0caad3310199dec7d5, State: Running, Role: LEADER
I20260812 19:04:02.028366 32710 consensus_queue.cc:237] T 00000000000000000000000000000000 P f42d7998d04a4c0caad3310199dec7d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f42d7998d04a4c0caad3310199dec7d5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42523 } }
I20260812 19:04:02.028952 32705 sys_catalog.cc:564] T 00000000000000000000000000000000 P f42d7998d04a4c0caad3310199dec7d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:04:02.029536 32710 sys_catalog.cc:455] T 00000000000000000000000000000000 P f42d7998d04a4c0caad3310199dec7d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f42d7998d04a4c0caad3310199dec7d5. Latest consensus state: current_term: 1 leader_uuid: "f42d7998d04a4c0caad3310199dec7d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f42d7998d04a4c0caad3310199dec7d5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42523 } } }
I20260812 19:04:02.029657 32710 sys_catalog.cc:458] T 00000000000000000000000000000000 P f42d7998d04a4c0caad3310199dec7d5 [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:02.030043 32712 sys_catalog.cc:455] T 00000000000000000000000000000000 P f42d7998d04a4c0caad3310199dec7d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f42d7998d04a4c0caad3310199dec7d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f42d7998d04a4c0caad3310199dec7d5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42523 } } }
I20260812 19:04:02.030045 32715 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:04:02.030210 32712 sys_catalog.cc:458] T 00000000000000000000000000000000 P f42d7998d04a4c0caad3310199dec7d5 [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:02.033102 32715 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:04:02.039099 32715 catalog_manager.cc:1347] Generated new cluster ID: a55d2c9199b944acb01a70f8c6f2bfbd
I20260812 19:04:02.039192 32715 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:04:02.047139 32715 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:04:02.048444 32715 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:04:02.053628 32715 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f42d7998d04a4c0caad3310199dec7d5: Generated new TSK 0
I20260812 19:04:02.054373 32715 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 19:04:02.097357 32740 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 19:04:02.097680 32740 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 19:04:02.098202 32708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:02.074665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58126 (local address 127.25.130.62:42523)
0812 19:04:02.074824 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:02.074829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:02.075099 (+ 270us) server_negotiation.cc:407] Connection header received
0812 19:04:02.075341 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:02.075343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:02.075401 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:02.075491 (+ 90us) server_negotiation.cc:226] Negotiated authn=JWT
0812 19:04:02.075794 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:02.076399 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:02.077708 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:02.077952 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:02.078039 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:02.078484 (+ 445us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 19:04:02.097675 (+ 19191us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36731/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 19:04:02.098063 (+ 388us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58126: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36731/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":58}
W20260812 19:04:02.098299 32708 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58126: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36731/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 19:04:02.100014 32739 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:02.074529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:42523 (local address 127.0.0.1:58126)
0812 19:04:02.074971 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:02.074988 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:02.075151 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:02.075528 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:02.075531 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:02.075541 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0812 19:04:02.075682 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:02.075689 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:02.076523 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:02.076528 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:02.077553 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:02.077569 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:02.078078 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:02.078082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:02.078185 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:02.078268 (+ 83us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 19:04:02.098036 (+ 19768us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36731/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 19:04:02.098342 (+ 306us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.25.130.62:42523: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36731/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":120,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.25.130.62:42523: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36731/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 19:04:02.100834 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 32620
2026-08-12T19:04:02Z chronyd exiting
I20260812 19:04:02.142411 26120 test_util.cc:175] -----------------------------------------------
I20260812 19:04:02.142511 26120 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561415988404-26120-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (338 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 19:04:02.143728 26120 mini_oidc.cc:152] Starting JWKS server
I20260812 19:04:02.144634 26120 webserver.cc:460] Webserver started at https://127.0.0.1:33249/ using document root <none> and password file <none>
I20260812 19:04:02.144750 26120 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 19:04:02.145025 26120 webserver.cc:460] Webserver started at http://127.0.0.1:40423/ using document root <none> and password file <none>
2026-08-12T19:04:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:04:02Z Disabled control of system clock
I20260812 19:04:02.155664 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:38141
--webserver_interface=127.25.130.62
--webserver_port=0
--builtin_ntp_servers=127.25.130.20:40917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:38141
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33249/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561415988404-26120-0/testchainca.cert with env {}
W20260812 19:04:02.272976 32748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:02.273315 32748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:02.273377 32748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:02.277678 32748 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:04:02.277781 32748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:02.277802 32748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:02.277825 32748 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:04:02.277844 32748 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 19:04:02.277868 32748 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 19:04:02.277894 32748 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33249/jwks.json
I20260812 19:04:02.283944 32748 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:40917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:38141
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:38141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33249/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561415988404-26120-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:02.285195 32748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:02.286350 32748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:02.295639 32756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:02.295701 32748 server_base.cc:1034] running on GCE node
W20260812 19:04:02.295640 32755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:02.295643 32758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:02.296577 32748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:02.297930 32748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:02.299113 32748 hybrid_clock.cc:648] HybridClock initialized: now 1786561442299096 us; error 40 us; skew 500 ppm
I20260812 19:04:02.301257 32748 webserver.cc:460] Webserver started at http://127.25.130.62:44065/ using document root <none> and password file <none>
I20260812 19:04:02.301962 32748 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:02.302037 32748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:02.302265 32748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:02.304571 32748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "db021d1a5fd94fda88d3a7747be830d7"
format_stamp: "Formatted at 2026-08-12 19:04:02 on dist-test-slave-49h3"
server_key: "4df6fbe0044ffea28411e49bc0de656f"
server_key_iv: "bb9d9095c49b95fbd528a1a41a40dbbb"
server_key_version: "encryptionkey@0"
I20260812 19:04:02.305228 32748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "db021d1a5fd94fda88d3a7747be830d7"
format_stamp: "Formatted at 2026-08-12 19:04:02 on dist-test-slave-49h3"
server_key: "4df6fbe0044ffea28411e49bc0de656f"
server_key_iv: "bb9d9095c49b95fbd528a1a41a40dbbb"
server_key_version: "encryptionkey@0"
I20260812 19:04:02.309607 32748 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 19:04:02.312536 32764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:02.314011 32748 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:04:02.314157 32748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "db021d1a5fd94fda88d3a7747be830d7"
format_stamp: "Formatted at 2026-08-12 19:04:02 on dist-test-slave-49h3"
server_key: "4df6fbe0044ffea28411e49bc0de656f"
server_key_iv: "bb9d9095c49b95fbd528a1a41a40dbbb"
server_key_version: "encryptionkey@0"
I20260812 19:04:02.314299 32748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:02.346865 32748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:02.347689 32748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:02.347903 32748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:02.355585 32748 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:38141
I20260812 19:04:02.355608 360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:38141 every 8 connection(s)
I20260812 19:04:02.356640 32748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:04:02.360106 361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:02.363088 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 32748
I20260812 19:04:02.363232 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:04:02.363550 26120 external_mini_cluster.cc:1428] Setting key 67dcd1ca2e65d488ae3bceb1eaf44f45
I20260812 19:04:02.366930 361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db021d1a5fd94fda88d3a7747be830d7: Bootstrap starting.
I20260812 19:04:02.369709 361 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db021d1a5fd94fda88d3a7747be830d7: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:02.370714 361 log.cc:824] T 00000000000000000000000000000000 P db021d1a5fd94fda88d3a7747be830d7: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:02.373000 361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db021d1a5fd94fda88d3a7747be830d7: No bootstrap required, opened a new log
I20260812 19:04:02.377291 361 raft_consensus.cc:348] T 00000000000000000000000000000000 P db021d1a5fd94fda88d3a7747be830d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db021d1a5fd94fda88d3a7747be830d7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 38141 } }
I20260812 19:04:02.377681 361 raft_consensus.cc:374] T 00000000000000000000000000000000 P db021d1a5fd94fda88d3a7747be830d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:02.377743 361 raft_consensus.cc:729] T 00000000000000000000000000000000 P db021d1a5fd94fda88d3a7747be830d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db021d1a5fd94fda88d3a7747be830d7, State: Initialized, Role: FOLLOWER
I20260812 19:04:02.378273 361 consensus_queue.cc:260] T 00000000000000000000000000000000 P db021d1a5fd94fda88d3a7747be830d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db021d1a5fd94fda88d3a7747be830d7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 38141 } }
I20260812 19:04:02.378446 361 raft_consensus.cc:388] T 00000000000000000000000000000000 P db021d1a5fd94fda88d3a7747be830d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:04:02.378517 361 raft_consensus.cc:482] T 00000000000000000000000000000000 P db021d1a5fd94fda88d3a7747be830d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:04:02.378618 361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db021d1a5fd94fda88d3a7747be830d7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:02.378685 364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:02.365106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59070 (local address 127.25.130.62:38141)
0812 19:04:02.365961 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:02.365977 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:02.366035 (+ 58us) server_negotiation.cc:407] Connection header received
0812 19:04:02.366740 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:02.366759 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:02.367239 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:02.367560 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:02.367949 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:02.369003 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:02.370303 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:02.370870 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:02.371036 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:02.376037 (+ 5001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:02.376047 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:02.376073 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:04:02.376729 (+ 656us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:02.376824 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:04:02.376858 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:02.376986 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:02.377203 (+ 217us) server_negotiation.cc:299] Negotiation successful
0812 19:04:02.377569 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":188,"threads_started":1}
I20260812 19:04:02.379748 361 raft_consensus.cc:504] T 00000000000000000000000000000000 P db021d1a5fd94fda88d3a7747be830d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db021d1a5fd94fda88d3a7747be830d7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 38141 } }
I20260812 19:04:02.380151 361 leader_election.cc:302] T 00000000000000000000000000000000 P db021d1a5fd94fda88d3a7747be830d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db021d1a5fd94fda88d3a7747be830d7; no voters:
I20260812 19:04:02.380527 361 leader_election.cc:288] T 00000000000000000000000000000000 P db021d1a5fd94fda88d3a7747be830d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:04:02.380632 366 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db021d1a5fd94fda88d3a7747be830d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:04:02.380899 366 raft_consensus.cc:686] T 00000000000000000000000000000000 P db021d1a5fd94fda88d3a7747be830d7 [term 1 LEADER]: Becoming Leader. State: Replica: db021d1a5fd94fda88d3a7747be830d7, State: Running, Role: LEADER
I20260812 19:04:02.381261 366 consensus_queue.cc:237] T 00000000000000000000000000000000 P db021d1a5fd94fda88d3a7747be830d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db021d1a5fd94fda88d3a7747be830d7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 38141 } }
I20260812 19:04:02.381990 361 sys_catalog.cc:564] T 00000000000000000000000000000000 P db021d1a5fd94fda88d3a7747be830d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:04:02.382294 26120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:04:02.382921 367 sys_catalog.cc:455] T 00000000000000000000000000000000 P db021d1a5fd94fda88d3a7747be830d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db021d1a5fd94fda88d3a7747be830d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db021d1a5fd94fda88d3a7747be830d7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 38141 } } }
I20260812 19:04:02.383070 367 sys_catalog.cc:458] T 00000000000000000000000000000000 P db021d1a5fd94fda88d3a7747be830d7 [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:02.383416 369 sys_catalog.cc:455] T 00000000000000000000000000000000 P db021d1a5fd94fda88d3a7747be830d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db021d1a5fd94fda88d3a7747be830d7. Latest consensus state: current_term: 1 leader_uuid: "db021d1a5fd94fda88d3a7747be830d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db021d1a5fd94fda88d3a7747be830d7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 38141 } } }
I20260812 19:04:02.383527 369 sys_catalog.cc:458] T 00000000000000000000000000000000 P db021d1a5fd94fda88d3a7747be830d7 [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:02.384234 379 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:04:02.388077 379 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:04:02.394656 379 catalog_manager.cc:1347] Generated new cluster ID: ea1e3803da93481aabd8dbeac379472c
I20260812 19:04:02.394778 379 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:04:02.417654 379 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:04:02.418736 379 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:04:02.428718 379 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db021d1a5fd94fda88d3a7747be830d7: Generated new TSK 0
I20260812 19:04:02.429551 379 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 19:04:02.468747 398 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 19:04:02.468850 398 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 19:04:02.469620 364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:02.446609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59086 (local address 127.25.130.62:38141)
0812 19:04:02.446736 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:02.446741 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:02.446945 (+ 204us) server_negotiation.cc:407] Connection header received
0812 19:04:02.447153 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:02.447156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:02.447208 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:02.447306 (+ 98us) server_negotiation.cc:226] Negotiated authn=JWT
0812 19:04:02.447560 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:02.448118 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:02.448997 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:02.449223 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:02.449310 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:02.449670 (+ 360us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 19:04:02.469105 (+ 19435us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33249/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 19:04:02.469411 (+ 306us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59086: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33249/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":43}
W20260812 19:04:02.469710 364 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59086: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33249/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260812 19:04:02.469755 397 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:02.446540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:38141 (local address 127.0.0.1:59086)
0812 19:04:02.446859 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:02.446871 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:02.446985 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:02.447319 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:02.447322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:02.447330 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0812 19:04:02.447449 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:02.447456 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:02.448240 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:02.448244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:02.448874 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:02.448881 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:02.449335 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:02.449339 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:02.449430 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:02.449491 (+ 61us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 19:04:02.469341 (+ 19850us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33249/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 19:04:02.469577 (+ 236us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.25.130.62:38141: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33249/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":122,"threads_started":1}
I20260812 19:04:02.470453 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 32748
2026-08-12T19:04:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (370 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 19:04:02.513742 26120 mini_oidc.cc:152] Starting JWKS server
I20260812 19:04:02.514608 26120 webserver.cc:460] Webserver started at https://127.0.0.1:41241/ using document root <none> and password file <none>
I20260812 19:04:02.514704 26120 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 19:04:02.514921 26120 webserver.cc:460] Webserver started at http://127.0.0.1:36369/ using document root <none> and password file <none>
2026-08-12T19:04:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:04:02Z Disabled control of system clock
I20260812 19:04:02.525367 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:46603
--webserver_interface=127.25.130.62
--webserver_port=0
--builtin_ntp_servers=127.25.130.20:42713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:46603
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41241/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 19:04:02.641271 405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:02.641615 405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:02.641659 405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:02.645828 405 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:04:02.645907 405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:02.645921 405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:02.645937 405 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:04:02.645951 405 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 19:04:02.645965 405 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 19:04:02.645978 405 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41241/jwks.json
I20260812 19:04:02.651124 405 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:42713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:46603
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:46603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41241/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:02.652338 405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:02.653805 405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:04:02.663223 405 server_base.cc:1034] running on GCE node
W20260812 19:04:02.663175 411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:02.663175 415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:02.663178 413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:02.664067 405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:02.665347 405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:02.666563 405 hybrid_clock.cc:648] HybridClock initialized: now 1786561442666531 us; error 48 us; skew 500 ppm
I20260812 19:04:02.668743 405 webserver.cc:460] Webserver started at http://127.25.130.62:34759/ using document root <none> and password file <none>
I20260812 19:04:02.669394 405 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:02.669515 405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:02.669759 405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:02.672212 405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "b7d141fd34a34749bcbae57ceb626e31"
format_stamp: "Formatted at 2026-08-12 19:04:02 on dist-test-slave-49h3"
server_key: "aef086e603777c84ae88c8cea06518ac"
server_key_iv: "f29da0f70b656c8b36d66f9b3b186eb9"
server_key_version: "encryptionkey@0"
I20260812 19:04:02.672914 405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "b7d141fd34a34749bcbae57ceb626e31"
format_stamp: "Formatted at 2026-08-12 19:04:02 on dist-test-slave-49h3"
server_key: "aef086e603777c84ae88c8cea06518ac"
server_key_iv: "f29da0f70b656c8b36d66f9b3b186eb9"
server_key_version: "encryptionkey@0"
I20260812 19:04:02.677646 405 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.003s
I20260812 19:04:02.680613 423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:02.682150 405 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:04:02.682320 405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "b7d141fd34a34749bcbae57ceb626e31"
format_stamp: "Formatted at 2026-08-12 19:04:02 on dist-test-slave-49h3"
server_key: "aef086e603777c84ae88c8cea06518ac"
server_key_iv: "f29da0f70b656c8b36d66f9b3b186eb9"
server_key_version: "encryptionkey@0"
I20260812 19:04:02.682447 405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:02.699824 405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:02.700637 405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:02.700846 405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:02.708416 405 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:46603
I20260812 19:04:02.708444 491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:46603 every 8 connection(s)
I20260812 19:04:02.709614 405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:04:02.712651 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 405
I20260812 19:04:02.712791 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:04:02.713069 26120 external_mini_cluster.cc:1428] Setting key 84daaccc295d56ae84a2e2e48a4f3286
I20260812 19:04:02.713310 492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:02.721258 492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7d141fd34a34749bcbae57ceb626e31: Bootstrap starting.
I20260812 19:04:02.724305 492 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7d141fd34a34749bcbae57ceb626e31: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:02.724476 496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:02.714608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37712 (local address 127.25.130.62:46603)
0812 19:04:02.715534 (+ 926us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:02.715556 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:02.715604 (+ 48us) server_negotiation.cc:407] Connection header received
0812 19:04:02.716358 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:02.716385 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:02.716949 (+ 564us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:02.717355 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:02.717979 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:02.719462 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:02.720463 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:02.721180 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:02.721347 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:02.722302 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:02.722309 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:02.722327 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:04:02.722869 (+ 542us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:02.722948 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:04:02.722977 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:02.723081 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:02.723167 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 19:04:02.723457 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":266,"threads_started":1}
I20260812 19:04:02.725194 492 log.cc:824] T 00000000000000000000000000000000 P b7d141fd34a34749bcbae57ceb626e31: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:02.727519 26120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:04:02.727672 492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7d141fd34a34749bcbae57ceb626e31: No bootstrap required, opened a new log
I20260812 19:04:02.731778 492 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7d141fd34a34749bcbae57ceb626e31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7d141fd34a34749bcbae57ceb626e31" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46603 } }
I20260812 19:04:02.732098 492 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7d141fd34a34749bcbae57ceb626e31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:02.732161 492 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7d141fd34a34749bcbae57ceb626e31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7d141fd34a34749bcbae57ceb626e31, State: Initialized, Role: FOLLOWER
I20260812 19:04:02.732653 492 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7d141fd34a34749bcbae57ceb626e31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7d141fd34a34749bcbae57ceb626e31" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46603 } }
I20260812 19:04:02.732801 492 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7d141fd34a34749bcbae57ceb626e31 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:04:02.732867 492 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7d141fd34a34749bcbae57ceb626e31 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:04:02.732973 492 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7d141fd34a34749bcbae57ceb626e31 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:02.734333 492 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7d141fd34a34749bcbae57ceb626e31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7d141fd34a34749bcbae57ceb626e31" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46603 } }
I20260812 19:04:02.734717 492 leader_election.cc:302] T 00000000000000000000000000000000 P b7d141fd34a34749bcbae57ceb626e31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7d141fd34a34749bcbae57ceb626e31; no voters:
I20260812 19:04:02.735072 492 leader_election.cc:288] T 00000000000000000000000000000000 P b7d141fd34a34749bcbae57ceb626e31 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:04:02.735198 498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7d141fd34a34749bcbae57ceb626e31 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:04:02.735446 498 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7d141fd34a34749bcbae57ceb626e31 [term 1 LEADER]: Becoming Leader. State: Replica: b7d141fd34a34749bcbae57ceb626e31, State: Running, Role: LEADER
I20260812 19:04:02.735790 498 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7d141fd34a34749bcbae57ceb626e31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7d141fd34a34749bcbae57ceb626e31" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46603 } }
I20260812 19:04:02.736456 492 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7d141fd34a34749bcbae57ceb626e31 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:04:02.737921 499 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7d141fd34a34749bcbae57ceb626e31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7d141fd34a34749bcbae57ceb626e31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7d141fd34a34749bcbae57ceb626e31" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46603 } } }
I20260812 19:04:02.738077 499 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7d141fd34a34749bcbae57ceb626e31 [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:02.738941 511 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:04:02.738917 500 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7d141fd34a34749bcbae57ceb626e31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7d141fd34a34749bcbae57ceb626e31. Latest consensus state: current_term: 1 leader_uuid: "b7d141fd34a34749bcbae57ceb626e31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7d141fd34a34749bcbae57ceb626e31" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46603 } } }
I20260812 19:04:02.739017 500 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7d141fd34a34749bcbae57ceb626e31 [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:02.741961 511 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:04:02.747862 511 catalog_manager.cc:1347] Generated new cluster ID: 1e36f163f3ad45e5bc2553f0262e5342
I20260812 19:04:02.747969 511 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:04:02.761358 511 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:04:02.762781 511 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:04:02.771922 511 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7d141fd34a34749bcbae57ceb626e31: Generated new TSK 0
I20260812 19:04:02.772920 511 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:04:02.795908 496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:02.790008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37728 (local address 127.25.130.62:46603)
0812 19:04:02.790162 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:02.790168 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:02.790337 (+ 169us) server_negotiation.cc:407] Connection header received
0812 19:04:02.790518 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:02.790522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:02.790608 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:02.790754 (+ 146us) server_negotiation.cc:226] Negotiated authn=JWT
0812 19:04:02.791049 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:02.791902 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:02.792880 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:02.793241 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:02.793350 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:02.793749 (+ 399us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 19:04:02.795442 (+ 1693us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41241/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 19:04:02.795745 (+ 303us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37728: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41241/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":54}
W20260812 19:04:02.795996 496 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37728: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41241/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260812 19:04:02.797611 526 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:02.789883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46603 (local address 127.0.0.1:37728)
0812 19:04:02.790242 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:02.790256 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:02.790374 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:02.790794 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:02.790797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:02.790806 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0812 19:04:02.790928 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:02.790933 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:02.792075 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:02.792079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:02.792756 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:02.792766 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:02.793394 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:02.793397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:02.793515 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:02.793549 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 19:04:02.795671 (+ 2122us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41241/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 19:04:02.797372 (+ 1701us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.25.130.62:46603: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41241/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":1430,"thread_start_us":126,"threads_started":1}
I20260812 19:04:02.798568 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 405
E20260812 19:04:02.809901 529 webserver.cc:533] Webserver: sslize failed:
E20260812 19:04:02.810017 529 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-12T19:04:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (327 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T19:04:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:04:02Z Disabled control of system clock
I20260812 19:04:02.851933 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:37541
--webserver_interface=127.25.130.62
--webserver_port=0
--builtin_ntp_servers=127.25.130.20:44345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:37541
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561415988404-26120-0/testchainca.cert with env {}
W20260812 19:04:02.969775 533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:02.970096 533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:02.970140 533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:02.974409 533 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:04:02.974495 533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:02.974510 533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:02.974527 533 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:04:02.974541 533 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 19:04:02.974557 533 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 19:04:02.974570 533 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 19:04:02.980540 533 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:44345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:37541
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:37541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561415988404-26120-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:02.981753 533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:02.982834 533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:02.991851 539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:02.991835 543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:02.991825 533 server_base.cc:1034] running on GCE node
W20260812 19:04:02.991819 540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:02.992662 533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:02.993906 533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:02.995092 533 hybrid_clock.cc:648] HybridClock initialized: now 1786561442995079 us; error 31 us; skew 500 ppm
I20260812 19:04:02.997097 533 webserver.cc:460] Webserver started at http://127.25.130.62:38783/ using document root <none> and password file <none>
I20260812 19:04:02.997740 533 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:02.997816 533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:02.998029 533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:03.000233 533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "5c7881bcd7fc42f58d0b2eb417f8b69e"
format_stamp: "Formatted at 2026-08-12 19:04:02 on dist-test-slave-49h3"
server_key: "f70deca903f73b75ef1acf6299691ce9"
server_key_iv: "d3bc3b654d6564ceb555d714a951895d"
server_key_version: "encryptionkey@0"
I20260812 19:04:03.000821 533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "5c7881bcd7fc42f58d0b2eb417f8b69e"
format_stamp: "Formatted at 2026-08-12 19:04:02 on dist-test-slave-49h3"
server_key: "f70deca903f73b75ef1acf6299691ce9"
server_key_iv: "d3bc3b654d6564ceb555d714a951895d"
server_key_version: "encryptionkey@0"
I20260812 19:04:03.004684 533 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 19:04:03.007270 552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:03.008519 533 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:04:03.008646 533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "5c7881bcd7fc42f58d0b2eb417f8b69e"
format_stamp: "Formatted at 2026-08-12 19:04:02 on dist-test-slave-49h3"
server_key: "f70deca903f73b75ef1acf6299691ce9"
server_key_iv: "d3bc3b654d6564ceb555d714a951895d"
server_key_version: "encryptionkey@0"
I20260812 19:04:03.008755 533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:03.019541 533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:03.020323 533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:03.020514 533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:03.027840 533 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:37541
I20260812 19:04:03.027861 616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:37541 every 8 connection(s)
I20260812 19:04:03.028848 533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:04:03.031870 617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:03.037925 617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c7881bcd7fc42f58d0b2eb417f8b69e: Bootstrap starting.
I20260812 19:04:03.039094 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 533
I20260812 19:04:03.039217 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:04:03.039484 26120 external_mini_cluster.cc:1428] Setting key dd27c68329dd115fc530e548b34336c3
I20260812 19:04:03.040896 617 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c7881bcd7fc42f58d0b2eb417f8b69e: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:03.041760 617 log.cc:824] T 00000000000000000000000000000000 P 5c7881bcd7fc42f58d0b2eb417f8b69e: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:03.043867 617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c7881bcd7fc42f58d0b2eb417f8b69e: No bootstrap required, opened a new log
I20260812 19:04:03.047628 617 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c7881bcd7fc42f58d0b2eb417f8b69e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7881bcd7fc42f58d0b2eb417f8b69e" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37541 } }
I20260812 19:04:03.047916 617 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c7881bcd7fc42f58d0b2eb417f8b69e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:03.047972 617 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c7881bcd7fc42f58d0b2eb417f8b69e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c7881bcd7fc42f58d0b2eb417f8b69e, State: Initialized, Role: FOLLOWER
I20260812 19:04:03.048444 617 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c7881bcd7fc42f58d0b2eb417f8b69e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7881bcd7fc42f58d0b2eb417f8b69e" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37541 } }
I20260812 19:04:03.048588 617 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c7881bcd7fc42f58d0b2eb417f8b69e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:04:03.048656 617 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c7881bcd7fc42f58d0b2eb417f8b69e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:04:03.048760 617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c7881bcd7fc42f58d0b2eb417f8b69e [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:03.048962 622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:03.040593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58422 (local address 127.25.130.62:37541)
0812 19:04:03.041359 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:03.041374 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:03.041433 (+ 59us) server_negotiation.cc:407] Connection header received
0812 19:04:03.042213 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:03.042234 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:03.042611 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:03.042953 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:03.043399 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:03.044438 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:03.045354 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:03.045940 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:03.046119 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:03.046967 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:03.046973 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:03.046989 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:04:03.047418 (+ 429us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:03.047482 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:04:03.047503 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:03.047596 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:03.047752 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 19:04:03.048072 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":167,"threads_started":1}
I20260812 19:04:03.049832 617 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c7881bcd7fc42f58d0b2eb417f8b69e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7881bcd7fc42f58d0b2eb417f8b69e" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37541 } }
I20260812 19:04:03.050194 617 leader_election.cc:302] T 00000000000000000000000000000000 P 5c7881bcd7fc42f58d0b2eb417f8b69e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c7881bcd7fc42f58d0b2eb417f8b69e; no voters:
I20260812 19:04:03.050545 617 leader_election.cc:288] T 00000000000000000000000000000000 P 5c7881bcd7fc42f58d0b2eb417f8b69e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:04:03.050662 625 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c7881bcd7fc42f58d0b2eb417f8b69e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:04:03.050884 625 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c7881bcd7fc42f58d0b2eb417f8b69e [term 1 LEADER]: Becoming Leader. State: Replica: 5c7881bcd7fc42f58d0b2eb417f8b69e, State: Running, Role: LEADER
I20260812 19:04:03.051215 625 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c7881bcd7fc42f58d0b2eb417f8b69e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7881bcd7fc42f58d0b2eb417f8b69e" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37541 } }
I20260812 19:04:03.051921 617 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c7881bcd7fc42f58d0b2eb417f8b69e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:04:03.052460 26120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:04:03.053104 626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c7881bcd7fc42f58d0b2eb417f8b69e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c7881bcd7fc42f58d0b2eb417f8b69e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7881bcd7fc42f58d0b2eb417f8b69e" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37541 } } }
I20260812 19:04:03.053104 628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c7881bcd7fc42f58d0b2eb417f8b69e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c7881bcd7fc42f58d0b2eb417f8b69e. Latest consensus state: current_term: 1 leader_uuid: "5c7881bcd7fc42f58d0b2eb417f8b69e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7881bcd7fc42f58d0b2eb417f8b69e" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37541 } } }
I20260812 19:04:03.053263 628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c7881bcd7fc42f58d0b2eb417f8b69e [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:03.053263 626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c7881bcd7fc42f58d0b2eb417f8b69e [sys.catalog]: This master's current role is: LEADER
W20260812 19:04:03.057009 644 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5c7881bcd7fc42f58d0b2eb417f8b69e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:04:03.057119 644 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:04:03.057191 638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:04:03.058164 638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:04:03.064354 638 catalog_manager.cc:1347] Generated new cluster ID: fc72c5c8a255422d882c3d743a47015b
I20260812 19:04:03.064453 638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:04:03.086031 638 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:04:03.086985 638 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:04:03.096755 638 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c7881bcd7fc42f58d0b2eb417f8b69e: Generated new TSK 0
I20260812 19:04:03.097544 638 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:04:03.171442 622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:03.108185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58424 (local address 127.25.130.62:37541)
0812 19:04:03.108319 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:03.108324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:03.108462 (+ 138us) server_negotiation.cc:407] Connection header received
0812 19:04:03.108631 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:03.108633 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:03.108695 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:03.108785 (+ 90us) server_negotiation.cc:226] Negotiated authn=JWT
0812 19:04:03.109034 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:03.109626 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:03.110851 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:03.111067 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:03.111133 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:03.111465 (+ 332us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 19:04:03.171025 (+ 59560us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 19:04:03.171314 (+ 289us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58424: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":56}
W20260812 19:04:03.171532 622 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58424: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 19:04:03.171754 656 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:03.108055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37541 (local address 127.0.0.1:58424)
0812 19:04:03.108373 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:03.108388 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:03.108509 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:03.108800 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:03.108803 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:03.108811 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0812 19:04:03.108942 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:03.108949 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:03.109736 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:03.109740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:03.110744 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:03.110753 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:03.111178 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:03.111182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:03.111276 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:03.111308 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 19:04:03.171304 (+ 59996us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 19:04:03.171547 (+ 243us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.25.130.62:37541: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":113,"threads_started":1}
I20260812 19:04:03.172484 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 533
2026-08-12T19:04:03Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (358 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (129 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (130 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:04:03 dist-test-slave-49h3 krb5kdc[670](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:04:03 dist-test-slave-49h3 krb5kdc[670](info): set up 2 sockets
Aug 12 19:04:03 dist-test-slave-49h3 krb5kdc[670](info): commencing operation
krb5kdc: starting...
W20260812 19:04:04.495824 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:04:04 dist-test-slave-49h3 krb5kdc[670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:04:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:04:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:04:04.625842 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:34857
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:36405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:34857
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:04:04.741684 686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:04.741971 686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:04.742013 686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:04.746167 686 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:04:04.746248 686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:04.746266 686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:04.746289 686 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:04:04.746304 686 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:04:04.751935 686 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:36405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:34857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:34857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:04.753059 686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:04.754181 686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:04.763005 693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:04.763005 696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:04.763006 694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:04.763286 686 server_base.cc:1034] running on GCE node
I20260812 19:04:04.764061 686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:04.765347 686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:04.766523 686 hybrid_clock.cc:648] HybridClock initialized: now 1786561444766496 us; error 54 us; skew 500 ppm
Aug 12 19:04:04 dist-test-slave-49h3 krb5kdc[670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561444, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:04.769433 686 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:04:04.770484 686 webserver.cc:460] Webserver started at http://127.25.130.62:43595/ using document root <none> and password file <none>
I20260812 19:04:04.771070 686 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:04.771143 686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:04.771365 686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:04.773628 686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "6bdc88378dd04b8889cd861cee1e9807"
format_stamp: "Formatted at 2026-08-12 19:04:04 on dist-test-slave-49h3"
server_key: "5fbcf1c667d6694d8f18210dc33e536c"
server_key_iv: "71bb8f1d5694ac1d66be1d050fd0db53"
server_key_version: "encryptionkey@0"
I20260812 19:04:04.774250 686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "6bdc88378dd04b8889cd861cee1e9807"
format_stamp: "Formatted at 2026-08-12 19:04:04 on dist-test-slave-49h3"
server_key: "5fbcf1c667d6694d8f18210dc33e536c"
server_key_iv: "71bb8f1d5694ac1d66be1d050fd0db53"
server_key_version: "encryptionkey@0"
I20260812 19:04:04.778198 686 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 19:04:04.780812 705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:04.782208 686 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:04:04.782342 686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "6bdc88378dd04b8889cd861cee1e9807"
format_stamp: "Formatted at 2026-08-12 19:04:04 on dist-test-slave-49h3"
server_key: "5fbcf1c667d6694d8f18210dc33e536c"
server_key_iv: "71bb8f1d5694ac1d66be1d050fd0db53"
server_key_version: "encryptionkey@0"
I20260812 19:04:04.782464 686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:04.805191 686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:04.808485 686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:04.808701 686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:04.816071 686 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:34857
I20260812 19:04:04.816121 765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:34857 every 8 connection(s)
I20260812 19:04:04.817130 686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:04:04.820127 766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:04.823321 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 686
I20260812 19:04:04.823467 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:04:04.823779 26120 external_mini_cluster.cc:1428] Setting key 7596dbec4dfc4367a5320b27e9147946
I20260812 19:04:04.826638 766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bdc88378dd04b8889cd861cee1e9807: Bootstrap starting.
I20260812 19:04:04.829248 766 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6bdc88378dd04b8889cd861cee1e9807: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:04.830240 766 log.cc:824] T 00000000000000000000000000000000 P 6bdc88378dd04b8889cd861cee1e9807: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:04.832763 766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bdc88378dd04b8889cd861cee1e9807: No bootstrap required, opened a new log
Aug 12 19:04:04 dist-test-slave-49h3 krb5kdc[670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:04:04.836228 766 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bdc88378dd04b8889cd861cee1e9807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bdc88378dd04b8889cd861cee1e9807" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 34857 } }
I20260812 19:04:04.836488 766 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6bdc88378dd04b8889cd861cee1e9807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:04.836560 766 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bdc88378dd04b8889cd861cee1e9807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bdc88378dd04b8889cd861cee1e9807, State: Initialized, Role: FOLLOWER
I20260812 19:04:04.837051 766 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bdc88378dd04b8889cd861cee1e9807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bdc88378dd04b8889cd861cee1e9807" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 34857 } }
I20260812 19:04:04.837227 766 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6bdc88378dd04b8889cd861cee1e9807 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:04:04.837301 766 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bdc88378dd04b8889cd861cee1e9807 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:04:04.837405 766 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bdc88378dd04b8889cd861cee1e9807 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:04.838778 766 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bdc88378dd04b8889cd861cee1e9807 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bdc88378dd04b8889cd861cee1e9807" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 34857 } }
I20260812 19:04:04.839172 766 leader_election.cc:302] T 00000000000000000000000000000000 P 6bdc88378dd04b8889cd861cee1e9807 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6bdc88378dd04b8889cd861cee1e9807; no voters:
I20260812 19:04:04.839578 766 leader_election.cc:288] T 00000000000000000000000000000000 P 6bdc88378dd04b8889cd861cee1e9807 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:04:04.839722 771 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bdc88378dd04b8889cd861cee1e9807 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:04:04.839955 769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:04.825274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60018 (local address 127.25.130.62:34857)
0812 19:04:04.825786 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:04.825803 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:04.825841 (+ 38us) server_negotiation.cc:407] Connection header received
0812 19:04:04.826497 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:04.826515 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:04.826901 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:04.827195 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:04.828089 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:04.829123 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:04.830317 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:04.830953 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:04.831129 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:04.833874 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:04.833879 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:04.833893 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:04.833923 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:04.836411 (+ 2488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:04.836951 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:04.836957 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:04.836964 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:04.837141 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:04.837461 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:04.837465 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:04.837467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:04.838047 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:04.838274 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:04.838488 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 19:04:04.838832 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":193,"threads_started":1}
I20260812 19:04:04.839956 771 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bdc88378dd04b8889cd861cee1e9807 [term 1 LEADER]: Becoming Leader. State: Replica: 6bdc88378dd04b8889cd861cee1e9807, State: Running, Role: LEADER
I20260812 19:04:04.840366 771 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bdc88378dd04b8889cd861cee1e9807 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bdc88378dd04b8889cd861cee1e9807" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 34857 } }
I20260812 19:04:04.840876 766 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bdc88378dd04b8889cd861cee1e9807 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:04:04.842083 772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bdc88378dd04b8889cd861cee1e9807 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6bdc88378dd04b8889cd861cee1e9807" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bdc88378dd04b8889cd861cee1e9807" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 34857 } } }
I20260812 19:04:04.842216 772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bdc88378dd04b8889cd861cee1e9807 [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:04.842468 773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bdc88378dd04b8889cd861cee1e9807 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bdc88378dd04b8889cd861cee1e9807. Latest consensus state: current_term: 1 leader_uuid: "6bdc88378dd04b8889cd861cee1e9807" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bdc88378dd04b8889cd861cee1e9807" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 34857 } } }
I20260812 19:04:04.842579 773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bdc88378dd04b8889cd861cee1e9807 [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:04.842854 783 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:04:04.846580 783 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:04:04.852984 783 catalog_manager.cc:1347] Generated new cluster ID: 32ef040c0d81455282b0ad00d044b84a
I20260812 19:04:04.853067 783 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:04:04.861971 783 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:04:04.863391 783 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:04:04.871685 783 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6bdc88378dd04b8889cd861cee1e9807: Generated new TSK 0
I20260812 19:04:04.872625 783 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
I20260812 19:04:04.905176 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:0
--local_ip_for_outbound_sockets=127.25.130.1
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:34857
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:36405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:04:05.023887 798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:05.024207 798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:05.024271 798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:05.028523 798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:05.028607 798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:05.028693 798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:04:05.034364 798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:36405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:34857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:05.035539 798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:05.036605 798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:05.046672 805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:05.046676 807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:05.046643 809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:05.047024 798 server_base.cc:1034] running on GCE node
I20260812 19:04:05.047449 798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:05.048086 798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:05.049265 798 hybrid_clock.cc:648] HybridClock initialized: now 1786561445049246 us; error 36 us; skew 500 ppm
Aug 12 19:04:05 dist-test-slave-49h3 krb5kdc[670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561445, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:05.052211 798 init.cc:377] Logged in from keytab as kudu/127.25.130.1@KRBTEST.COM (short username kudu)
I20260812 19:04:05.053256 798 webserver.cc:460] Webserver started at http://127.25.130.1:40429/ using document root <none> and password file <none>
I20260812 19:04:05.053867 798 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:05.053943 798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:05.054167 798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:05.056324 798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-0/data/instance:
uuid: "5e1c21bad4934ca3bee46603bf96628b"
format_stamp: "Formatted at 2026-08-12 19:04:05 on dist-test-slave-49h3"
server_key: "19f7a3d51ccf3b7b4952f6a329db59a8"
server_key_iv: "bb3753e868c97a5ae5ba4fa25c1f8f93"
server_key_version: "encryptionkey@0"
I20260812 19:04:05.056915 798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance:
uuid: "5e1c21bad4934ca3bee46603bf96628b"
format_stamp: "Formatted at 2026-08-12 19:04:05 on dist-test-slave-49h3"
server_key: "19f7a3d51ccf3b7b4952f6a329db59a8"
server_key_iv: "bb3753e868c97a5ae5ba4fa25c1f8f93"
server_key_version: "encryptionkey@0"
I20260812 19:04:05.061144 798 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 19:04:05.064114 818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:05.065423 798 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 19:04:05.065603 798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "5e1c21bad4934ca3bee46603bf96628b"
format_stamp: "Formatted at 2026-08-12 19:04:05 on dist-test-slave-49h3"
server_key: "19f7a3d51ccf3b7b4952f6a329db59a8"
server_key_iv: "bb3753e868c97a5ae5ba4fa25c1f8f93"
server_key_version: "encryptionkey@0"
I20260812 19:04:05.065730 798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:05.089370 798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:05.092757 798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:05.092947 798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:05.093601 798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:05.094432 798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:05.094481 798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:05.094520 798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:05.094540 798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:05.103322 798 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:38001
I20260812 19:04:05.103348 965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:38001 every 8 connection(s)
I20260812 19:04:05.104267 798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
Aug 12 19:04:05 dist-test-slave-49h3 krb5kdc[670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561445, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:04:05.113574 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 798
I20260812 19:04:05.113687 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance
I20260812 19:04:05.113955 26120 external_mini_cluster.cc:1428] Setting key 33dd89ff36e511516378dc8903f17382
I20260812 19:04:05.118040 769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:05.106120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:35565 (local address 127.25.130.62:34857)
0812 19:04:05.106259 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:05.106262 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:05.107110 (+ 848us) server_negotiation.cc:407] Connection header received
0812 19:04:05.108459 (+ 1349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:05.108462 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:05.108511 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:05.108585 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:05.109815 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:05.110342 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:05.111444 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:05.111672 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:05.111744 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:05.114272 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:05.114276 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:05.114278 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:05.114307 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:05.115897 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:05.116432 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:05.116435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:05.116437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:05.116533 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:05.116831 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:05.116834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:05.116835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:05.117052 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:05.117149 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:05.117713 (+ 564us) server_negotiation.cc:299] Negotiation successful
0812 19:04:05.117867 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 19:04:05.118988 969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:05.106500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:34857 (local address 127.25.130.1:35565)
0812 19:04:05.106954 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:05.106999 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:05.108260 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:05.108750 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:05.108757 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:05.109126 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:05.109692 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:05.109709 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:05.110438 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:05.110440 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:05.111348 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:05.111358 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:05.111798 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:05.111800 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:05.111950 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:05.112523 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:05.112540 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:05.114073 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:05.116036 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:05.116041 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:05.116055 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:05.116337 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:05.116618 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:05.116620 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:05.116622 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:05.116756 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:05.117153 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:05.117159 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:05.117424 (+ 265us) client_negotiation.cc:769] Sending connection context
0812 19:04:05.117649 (+ 225us) client_negotiation.cc:240] Negotiation successful
0812 19:04:05.118004 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":119,"threads_started":1}
I20260812 19:04:05.120249 966 heartbeater.cc:347] Connected to a master server at 127.25.130.62:34857
I20260812 19:04:05.120549 966 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:05.121203 966 heartbeater.cc:511] Master 127.25.130.62:34857 requested a full tablet report, sending...
I20260812 19:04:05.123087 725 ts_manager.cc:194] Registered new tserver with Master: 5e1c21bad4934ca3bee46603bf96628b (127.25.130.1:38001)
WARNING: no policy specified for kudu/127.25.130.2@KRBTEST.COM; defaulting to no policy
I20260812 19:04:05.124775 725 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.1@KRBTEST.COM'} at 127.25.130.1:35565
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
I20260812 19:04:05.142613 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:0
--local_ip_for_outbound_sockets=127.25.130.2
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:34857
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:36405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:04:05.256480 973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:05.256778 973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:05.256824 973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:05.261015 973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:05.261106 973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:05.261179 973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:04:05.266812 973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:36405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:34857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:05.267951 973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:05.269022 973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:05.278981 979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:05.278993 980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:05.278981 982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:05.279027 973 server_base.cc:1034] running on GCE node
I20260812 19:04:05.279621 973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:05.280308 973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:05.281520 973 hybrid_clock.cc:648] HybridClock initialized: now 1786561445281492 us; error 43 us; skew 500 ppm
Aug 12 19:04:05 dist-test-slave-49h3 krb5kdc[670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561445, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:05.284310 973 init.cc:377] Logged in from keytab as kudu/127.25.130.2@KRBTEST.COM (short username kudu)
I20260812 19:04:05.285259 973 webserver.cc:460] Webserver started at http://127.25.130.2:46797/ using document root <none> and password file <none>
I20260812 19:04:05.285852 973 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:05.285923 973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:05.286132 973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:05.288421 973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-1/data/instance:
uuid: "91478f329bb543b59141234bdc39c77f"
format_stamp: "Formatted at 2026-08-12 19:04:05 on dist-test-slave-49h3"
server_key: "1f6b7de308618bb90e84ca96a17c3766"
server_key_iv: "84edeb35fbc73a39b15496f7d1844cc3"
server_key_version: "encryptionkey@0"
I20260812 19:04:05.289017 973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance:
uuid: "91478f329bb543b59141234bdc39c77f"
format_stamp: "Formatted at 2026-08-12 19:04:05 on dist-test-slave-49h3"
server_key: "1f6b7de308618bb90e84ca96a17c3766"
server_key_iv: "84edeb35fbc73a39b15496f7d1844cc3"
server_key_version: "encryptionkey@0"
I20260812 19:04:05.292865 973 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 19:04:05.295429 992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:05.296686 973 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:04:05.296809 973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "91478f329bb543b59141234bdc39c77f"
format_stamp: "Formatted at 2026-08-12 19:04:05 on dist-test-slave-49h3"
server_key: "1f6b7de308618bb90e84ca96a17c3766"
server_key_iv: "84edeb35fbc73a39b15496f7d1844cc3"
server_key_version: "encryptionkey@0"
I20260812 19:04:05.296916 973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:05.313023 973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:05.315874 973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:05.316073 973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:05.316680 973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:05.317595 973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:05.317647 973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:05.317704 973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:05.317730 973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:05.326583 973 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:38411
I20260812 19:04:05.326606 1126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:38411 every 8 connection(s)
I20260812 19:04:05.327519 973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:04:05.329736 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 973
I20260812 19:04:05.329859 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance
I20260812 19:04:05.330145 26120 external_mini_cluster.cc:1428] Setting key 354157c9224ba19324aee0bc8b561d4c
Aug 12 19:04:05 dist-test-slave-49h3 krb5kdc[670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561445, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.130.3@KRBTEST.COM; defaulting to no policy
I20260812 19:04:05.342568 769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:05.329376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:52703 (local address 127.25.130.62:34857)
0812 19:04:05.329546 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:05.329551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:05.330536 (+ 985us) server_negotiation.cc:407] Connection header received
0812 19:04:05.332102 (+ 1566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:05.332106 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:05.332159 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:05.332244 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:05.333478 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:05.334055 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:05.335672 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:05.335905 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:05.335996 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:05.339127 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:05.339130 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:05.339132 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:05.339161 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:05.340485 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:05.341012 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:05.341015 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:05.341017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:05.341106 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:05.341403 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:05.341405 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:05.341406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:05.341667 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:05.341769 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:05.342310 (+ 541us) server_negotiation.cc:299] Negotiation successful
0812 19:04:05.342436 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.3@KRBTEST.COM" created.
I20260812 19:04:05.343499 1132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:05.329793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:34857 (local address 127.25.130.2:52703)
0812 19:04:05.330351 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:05.330416 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:05.331874 (+ 1458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:05.332402 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:05.332409 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:05.332784 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:05.333288 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:05.333303 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:05.334199 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:05.334203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:05.335522 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:05.335534 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:05.336075 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:05.336081 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:05.336312 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:05.336996 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:05.337021 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:05.338900 (+ 1879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:05.340627 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:05.340633 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:05.340646 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:05.340910 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:05.341184 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:05.341186 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:05.341188 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:05.341330 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:05.341772 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:05.341779 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:05.342073 (+ 294us) client_negotiation.cc:769] Sending connection context
0812 19:04:05.342247 (+ 174us) client_negotiation.cc:240] Negotiation successful
0812 19:04:05.342560 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":118,"threads_started":1}
I20260812 19:04:05.344785 1129 heartbeater.cc:347] Connected to a master server at 127.25.130.62:34857
I20260812 19:04:05.345027 1129 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:05.345594 1129 heartbeater.cc:511] Master 127.25.130.62:34857 requested a full tablet report, sending...
I20260812 19:04:05.346678 725 ts_manager.cc:194] Registered new tserver with Master: 91478f329bb543b59141234bdc39c77f (127.25.130.2:38411)
I20260812 19:04:05.347419 725 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.2@KRBTEST.COM'} at 127.25.130.2:52703
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
I20260812 19:04:05.359875 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:0
--local_ip_for_outbound_sockets=127.25.130.3
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:34857
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:36405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:04:05.475590 1135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:05.475862 1135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:05.475906 1135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:05.480214 1135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:05.480302 1135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:05.480369 1135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:04:05.486044 1135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:36405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:34857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:05.487377 1135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:05.488437 1135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:05.498446 1143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:05.498488 1135 server_base.cc:1034] running on GCE node
W20260812 19:04:05.498449 1145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:05.498471 1142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:05.499091 1135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:05.499827 1135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:05.501027 1135 hybrid_clock.cc:648] HybridClock initialized: now 1786561445501004 us; error 34 us; skew 500 ppm
Aug 12 19:04:05 dist-test-slave-49h3 krb5kdc[670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561445, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:05.503943 1135 init.cc:377] Logged in from keytab as kudu/127.25.130.3@KRBTEST.COM (short username kudu)
I20260812 19:04:05.504868 1135 webserver.cc:460] Webserver started at http://127.25.130.3:38113/ using document root <none> and password file <none>
I20260812 19:04:05.505409 1135 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:05.505511 1135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:05.505731 1135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:05.507879 1135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-2/data/instance:
uuid: "13ab7b8a1f3a4d04959be7256c5ed41b"
format_stamp: "Formatted at 2026-08-12 19:04:05 on dist-test-slave-49h3"
server_key: "944205848dac33022078f06c6faaddda"
server_key_iv: "cac75ac9d996a65d7c4757f091bbfce0"
server_key_version: "encryptionkey@0"
I20260812 19:04:05.508435 1135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance:
uuid: "13ab7b8a1f3a4d04959be7256c5ed41b"
format_stamp: "Formatted at 2026-08-12 19:04:05 on dist-test-slave-49h3"
server_key: "944205848dac33022078f06c6faaddda"
server_key_iv: "cac75ac9d996a65d7c4757f091bbfce0"
server_key_version: "encryptionkey@0"
I20260812 19:04:05.512440 1135 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260812 19:04:05.515077 1155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:05.516364 1135 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260812 19:04:05.516489 1135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "13ab7b8a1f3a4d04959be7256c5ed41b"
format_stamp: "Formatted at 2026-08-12 19:04:05 on dist-test-slave-49h3"
server_key: "944205848dac33022078f06c6faaddda"
server_key_iv: "cac75ac9d996a65d7c4757f091bbfce0"
server_key_version: "encryptionkey@0"
I20260812 19:04:05.516598 1135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:05.547712 1135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:05.551266 1135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:05.551491 1135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:05.552166 1135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:05.553110 1135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:05.553162 1135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:05.553221 1135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:05.553247 1135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:05.562611 1135 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:35009
I20260812 19:04:05.562685 1289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:35009 every 8 connection(s)
I20260812 19:04:05.563719 1135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:04:05.568146 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 1135
I20260812 19:04:05.568281 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance
I20260812 19:04:05.568614 26120 external_mini_cluster.cc:1428] Setting key be682faea78619280a52da464580f7f0
Aug 12 19:04:05 dist-test-slave-49h3 krb5kdc[670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561445, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:04:05.578719 769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:05.565794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:46587 (local address 127.25.130.62:34857)
0812 19:04:05.565976 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:05.565981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:05.567013 (+ 1032us) server_negotiation.cc:407] Connection header received
0812 19:04:05.568521 (+ 1508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:05.568525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:05.568595 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:05.568719 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:05.570337 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:05.570942 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:05.572266 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:05.572508 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:05.572605 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:05.575327 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:05.575331 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:05.575334 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:05.575365 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:05.576717 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:05.577224 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:05.577227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:05.577228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:05.577306 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:05.577652 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:05.577655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:05.577656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:05.577858 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:05.577936 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:05.578475 (+ 539us) server_negotiation.cc:299] Negotiation successful
0812 19:04:05.578587 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 19:04:05.579663 1293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:05.566263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:34857 (local address 127.25.130.3:46587)
0812 19:04:05.566831 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:05.566882 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:05.568215 (+ 1333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:05.568944 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:05.568953 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:05.569476 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:05.570187 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:05.570206 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:05.571081 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:05.571085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:05.572012 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:05.572024 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:05.572683 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:05.572688 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:05.572878 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:05.573531 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:05.573549 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:05.575118 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:05.576823 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:05.576827 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:05.576842 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:05.577116 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:05.577375 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:05.577378 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:05.577379 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:05.577572 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:05.577936 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:05.577942 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:05.578223 (+ 281us) client_negotiation.cc:769] Sending connection context
0812 19:04:05.578442 (+ 219us) client_negotiation.cc:240] Negotiation successful
0812 19:04:05.578744 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":143,"threads_started":1}
I20260812 19:04:05.580860 1290 heartbeater.cc:347] Connected to a master server at 127.25.130.62:34857
I20260812 19:04:05.581108 1290 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:05.581725 1290 heartbeater.cc:511] Master 127.25.130.62:34857 requested a full tablet report, sending...
I20260812 19:04:05.582815 724 ts_manager.cc:194] Registered new tserver with Master: 13ab7b8a1f3a4d04959be7256c5ed41b (127.25.130.3:35009)
I20260812 19:04:05.583426 724 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.3@KRBTEST.COM'} at 127.25.130.3:46587
I20260812 19:04:05.594841 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/test-admin.keytab.
Aug 12 19:04:05 dist-test-slave-49h3 krb5kdc[670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:05.610201 26120 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 19:04:05.613226 1301 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:05.611636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56470 (local address 127.25.130.1:38001)
0812 19:04:05.611962 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:05.611980 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:05.612002 (+ 22us) server_negotiation.cc:407] Connection header received
0812 19:04:05.612054 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:05.612059 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:05.612263 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:05.612392 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:05.613032 (+ 640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56470: BlockingRecv error: recv got EOF from 127.0.0.1:56470 (error 108)
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":101,"threads_started":1}
W20260812 19:04:05.613304 26120 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/krb5cc' of type FILE
Aug 12 19:04:05 dist-test-slave-49h3 krb5kdc[670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:05.614364 26120 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 19:04:05 dist-test-slave-49h3 krb5kdc[670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:05.615226 26120 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 19:04:05 dist-test-slave-49h3 krb5kdc[670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.1@KRBTEST.COM
I20260812 19:04:05.624811 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:05.616165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56476 (local address 127.25.130.1:38001)
0812 19:04:05.616298 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:05.616302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:05.616365 (+ 63us) server_negotiation.cc:407] Connection header received
0812 19:04:05.616474 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:05.616477 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:05.616515 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:05.616579 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:05.617070 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:05.617929 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:05.618808 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:05.619222 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:05.619308 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:05.621651 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:05.621663 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:05.621677 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:05.621707 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:05.623117 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:05.623551 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:05.623556 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:05.623557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:05.623640 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:05.623872 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:05.623874 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:05.623875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:05.624145 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:05.624305 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:05.624472 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 19:04:05.624633 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260812 19:04:05.628679 1301 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:05.627787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56490 (local address 127.25.130.1:38001)
0812 19:04:05.627912 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:05.627916 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:05.628007 (+ 91us) server_negotiation.cc:407] Connection header received
0812 19:04:05.628104 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:05.628107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:05.628154 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:05.628216 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:05.628568 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56490: BlockingRecv error: recv got EOF from 127.0.0.1:56490 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260812 19:04:05.628811 26120 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/krb5cc' of type FILE
Aug 12 19:04:05 dist-test-slave-49h3 krb5kdc[670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:05.629830 26120 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 19:04:05 dist-test-slave-49h3 krb5kdc[670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:05.630658 26120 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 19:04:05 dist-test-slave-49h3 krb5kdc[670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.1@KRBTEST.COM
I20260812 19:04:05.638789 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:05.631576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56496 (local address 127.25.130.1:38001)
0812 19:04:05.631701 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:05.631706 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:05.631785 (+ 79us) server_negotiation.cc:407] Connection header received
0812 19:04:05.631912 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:05.631915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:05.631955 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:05.632012 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:05.632461 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:05.632962 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:05.633763 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:05.633989 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:05.634064 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:05.636168 (+ 2104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:05.636171 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:05.636173 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:05.636201 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:05.637329 (+ 1128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:05.637802 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:05.637804 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:05.637806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:05.637881 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:05.638071 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:05.638073 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:05.638074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:05.638256 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:05.638375 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:05.638524 (+ 149us) server_negotiation.cc:299] Negotiation successful
0812 19:04:05.638627 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260812 19:04:05.641578 1301 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:05.640573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56500 (local address 127.25.130.1:38001)
0812 19:04:05.640681 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:05.640686 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:05.640816 (+ 130us) server_negotiation.cc:407] Connection header received
0812 19:04:05.640923 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:05.640926 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:05.640964 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:05.641025 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:05.641480 (+ 455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56500: BlockingRecv error: recv got EOF from 127.0.0.1:56500 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260812 19:04:05.641719 26120 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561415988404-26120-0/krb5kdc/krb5cc' of type FILE
Aug 12 19:04:05 dist-test-slave-49h3 krb5kdc[670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:05.642733 26120 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 19:04:05 dist-test-slave-49h3 krb5kdc[670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:05.643538 26120 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 19:04:05 dist-test-slave-49h3 krb5kdc[670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.1@KRBTEST.COM
I20260812 19:04:05.651820 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:05.644384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56510 (local address 127.25.130.1:38001)
0812 19:04:05.644503 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:05.644507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:05.644590 (+ 83us) server_negotiation.cc:407] Connection header received
0812 19:04:05.644718 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:05.644722 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:05.644763 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:05.644826 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:05.645329 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:05.645884 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:05.646684 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:05.646899 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:05.646987 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:05.649136 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:05.649140 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:05.649142 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:05.649168 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:05.650477 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:05.650863 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:05.650866 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:05.650867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:05.650939 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:05.651120 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:05.651122 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:05.651135 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:05.651297 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:05.651390 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:05.651557 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 19:04:05.651653 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 19:04:05.652927 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 798
I20260812 19:04:05.662207 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 973
I20260812 19:04:05.676834 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 1135
I20260812 19:04:05.689543 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 686
2026-08-12T19:04:05Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2254 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:04:05 dist-test-slave-49h3 krb5kdc[1315](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:04:05 dist-test-slave-49h3 krb5kdc[1315](info): set up 2 sockets
Aug 12 19:04:05 dist-test-slave-49h3 krb5kdc[1315](info): commencing operation
krb5kdc: starting...
W20260812 19:04:06.760865 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:04:06 dist-test-slave-49h3 krb5kdc[1315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:04:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:04:06Z Disabled control of system clock
WARNING: no policy specified for oryx/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.62.keytab.
Entry for principal oryx/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.62.keytab.
I20260812 19:04:06.889729 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:37435
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.62.keytab
--principal=oryx/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:45129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:37435
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:04:07.003867 1331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:07.004189 1331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:07.004247 1331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:07.008461 1331 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:04:07.008548 1331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:07.008577 1331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:07.008599 1331 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:04:07.008620 1331 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:04:07.014314 1331 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:45129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:37435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.62.keytab
--principal=oryx/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:37435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:07.015507 1331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:07.016601 1331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:07.025672 1338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:07.025672 1342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:07.025691 1339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:07.026077 1331 server_base.cc:1034] running on GCE node
I20260812 19:04:07.026690 1331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:07.027866 1331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:07.029078 1331 hybrid_clock.cc:648] HybridClock initialized: now 1786561447029063 us; error 32 us; skew 500 ppm
Aug 12 19:04:07 dist-test-slave-49h3 krb5kdc[1315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561447, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:07.031944 1331 init.cc:377] Logged in from keytab as oryx/127.25.130.62@KRBTEST.COM (short username oryx)
I20260812 19:04:07.032933 1331 webserver.cc:460] Webserver started at http://127.25.130.62:35609/ using document root <none> and password file <none>
I20260812 19:04:07.033537 1331 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:07.033615 1331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:07.033856 1331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:07.036037 1331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "dede9ac8627f4a27b237459ecc7d8ba3"
format_stamp: "Formatted at 2026-08-12 19:04:07 on dist-test-slave-49h3"
server_key: "5f40da1795f26cedddad19146c54f22d"
server_key_iv: "41243c8d0d9ae141787c28aeaed52926"
server_key_version: "encryptionkey@0"
I20260812 19:04:07.036629 1331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "dede9ac8627f4a27b237459ecc7d8ba3"
format_stamp: "Formatted at 2026-08-12 19:04:07 on dist-test-slave-49h3"
server_key: "5f40da1795f26cedddad19146c54f22d"
server_key_iv: "41243c8d0d9ae141787c28aeaed52926"
server_key_version: "encryptionkey@0"
I20260812 19:04:07.040531 1331 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 19:04:07.043092 1349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:07.044333 1331 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 19:04:07.044456 1331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "dede9ac8627f4a27b237459ecc7d8ba3"
format_stamp: "Formatted at 2026-08-12 19:04:07 on dist-test-slave-49h3"
server_key: "5f40da1795f26cedddad19146c54f22d"
server_key_iv: "41243c8d0d9ae141787c28aeaed52926"
server_key_version: "encryptionkey@0"
I20260812 19:04:07.044559 1331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:07.074031 1331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:07.077334 1331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:07.077564 1331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:07.084887 1331 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:37435
I20260812 19:04:07.085163 1419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:37435 every 8 connection(s)
I20260812 19:04:07.085970 1331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:04:07.087270 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 1331
I20260812 19:04:07.087412 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:04:07.087738 26120 external_mini_cluster.cc:1428] Setting key 756af03dbfd846c7f787333e467ed807
I20260812 19:04:07.089352 1420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:07.096185 1420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3: Bootstrap starting.
Aug 12 19:04:07 dist-test-slave-49h3 krb5kdc[1315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM
I20260812 19:04:07.098744 1420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:07.099570 1420 log.cc:824] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:07.101918 1420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3: No bootstrap required, opened a new log
I20260812 19:04:07.102306 1423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:07.089145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38624 (local address 127.25.130.62:37435)
0812 19:04:07.089583 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:07.089600 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:07.089646 (+ 46us) server_negotiation.cc:407] Connection header received
0812 19:04:07.090335 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:07.090352 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:07.090690 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:07.090968 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:07.091869 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:07.092888 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:07.094024 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:07.094498 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:07.094613 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:07.097498 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:07.097506 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:07.097523 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:07.097556 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:07.099268 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:07.099780 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:07.099786 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:07.099791 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:07.099913 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:07.100202 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:07.100205 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:07.100206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:07.100688 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:07.100848 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:07.101151 (+ 303us) server_negotiation.cc:299] Negotiation successful
0812 19:04:07.101473 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":118,"threads_started":1}
I20260812 19:04:07.106032 1420 raft_consensus.cc:348] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } }
I20260812 19:04:07.106289 1420 raft_consensus.cc:374] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:07.106341 1420 raft_consensus.cc:729] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dede9ac8627f4a27b237459ecc7d8ba3, State: Initialized, Role: FOLLOWER
I20260812 19:04:07.106775 1420 consensus_queue.cc:260] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } }
I20260812 19:04:07.106904 1420 raft_consensus.cc:388] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:04:07.106968 1420 raft_consensus.cc:482] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:04:07.107072 1420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:07.108181 1420 raft_consensus.cc:504] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } }
I20260812 19:04:07.108532 1420 leader_election.cc:302] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dede9ac8627f4a27b237459ecc7d8ba3; no voters:
I20260812 19:04:07.108863 1420 leader_election.cc:288] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:04:07.108978 1425 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:04:07.109233 1425 raft_consensus.cc:686] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [term 1 LEADER]: Becoming Leader. State: Replica: dede9ac8627f4a27b237459ecc7d8ba3, State: Running, Role: LEADER
I20260812 19:04:07.109575 1425 consensus_queue.cc:237] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } }
I20260812 19:04:07.110183 1420 sys_catalog.cc:564] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:04:07.111474 1428 sys_catalog.cc:455] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dede9ac8627f4a27b237459ecc7d8ba3. Latest consensus state: current_term: 1 leader_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } }
I20260812 19:04:07.111617 1428 sys_catalog.cc:458] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:07.112028 1435 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:04:07.112020 1427 sys_catalog.cc:455] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } }
I20260812 19:04:07.112192 1427 sys_catalog.cc:458] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:07.115197 1435 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:04:07.121472 1435 catalog_manager.cc:1347] Generated new cluster ID: a674a8bb57f4493194ed80aa887c9dca
I20260812 19:04:07.121565 1435 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:04:07.129509 1435 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:04:07.130625 1435 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:04:07.136283 1435 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3: Generated new TSK 0
I20260812 19:04:07.137060 1435 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.25.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.1.keytab.
Entry for principal oryx/127.25.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.1.keytab.
I20260812 19:04:07.172793 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:0
--local_ip_for_outbound_sockets=127.25.130.1
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:37435
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.1.keytab
--principal=oryx/127.25.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:45129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:04:07.293177 1453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:07.293644 1453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:07.293715 1453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:07.300132 1453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:07.300275 1453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:07.300370 1453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:04:07.308980 1453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:45129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.1.keytab
--principal=oryx/127.25.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:37435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:07.310537 1453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:07.311929 1453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:07.323819 1463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:07.323805 1460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:07.323805 1461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:07.324352 1453 server_base.cc:1034] running on GCE node
I20260812 19:04:07.324898 1453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:07.325670 1453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:07.326844 1453 hybrid_clock.cc:648] HybridClock initialized: now 1786561447326801 us; error 68 us; skew 500 ppm
Aug 12 19:04:07 dist-test-slave-49h3 krb5kdc[1315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561447, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:07.329818 1453 init.cc:377] Logged in from keytab as oryx/127.25.130.1@KRBTEST.COM (short username oryx)
I20260812 19:04:07.330842 1453 webserver.cc:460] Webserver started at http://127.25.130.1:37503/ using document root <none> and password file <none>
I20260812 19:04:07.331449 1453 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:07.331527 1453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:07.331754 1453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:07.334084 1453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/data/instance:
uuid: "7aa537f1f40c4f53abe5e8e8adec26dc"
format_stamp: "Formatted at 2026-08-12 19:04:07 on dist-test-slave-49h3"
server_key: "338e00ddb6ae993f06f5d229e39fad3d"
server_key_iv: "fbbe4626e91c9c42efd432358efe6845"
server_key_version: "encryptionkey@0"
I20260812 19:04:07.334723 1453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance:
uuid: "7aa537f1f40c4f53abe5e8e8adec26dc"
format_stamp: "Formatted at 2026-08-12 19:04:07 on dist-test-slave-49h3"
server_key: "338e00ddb6ae993f06f5d229e39fad3d"
server_key_iv: "fbbe4626e91c9c42efd432358efe6845"
server_key_version: "encryptionkey@0"
I20260812 19:04:07.339232 1453 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:04:07.342124 1472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:07.343653 1453 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:04:07.343811 1453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "7aa537f1f40c4f53abe5e8e8adec26dc"
format_stamp: "Formatted at 2026-08-12 19:04:07 on dist-test-slave-49h3"
server_key: "338e00ddb6ae993f06f5d229e39fad3d"
server_key_iv: "fbbe4626e91c9c42efd432358efe6845"
server_key_version: "encryptionkey@0"
I20260812 19:04:07.343942 1453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:07.392452 1453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:07.396767 1453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:07.397018 1453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:07.397873 1453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:07.398823 1453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:07.398876 1453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:07.398921 1453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:07.398941 1453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:07.408185 1453 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:37323
I20260812 19:04:07.408239 1612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:37323 every 8 connection(s)
I20260812 19:04:07.409185 1453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:04:07.409634 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 1453
I20260812 19:04:07.409768 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance
I20260812 19:04:07.410071 26120 external_mini_cluster.cc:1428] Setting key 19a42af79c84b3152cdff803c9b58717
Aug 12 19:04:07 dist-test-slave-49h3 krb5kdc[1315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561447, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM
WARNING: no policy specified for oryx/127.25.130.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.130.2@KRBTEST.COM" created.
I20260812 19:04:07.425537 1423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:07.411595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:33067 (local address 127.25.130.62:37435)
0812 19:04:07.411729 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:07.411735 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:07.412971 (+ 1236us) server_negotiation.cc:407] Connection header received
0812 19:04:07.414491 (+ 1520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:07.414494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:07.414564 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:07.414637 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:07.415996 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:07.416570 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:07.418249 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:07.418507 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:07.418610 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:07.421764 (+ 3154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:07.421768 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:07.421770 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:07.421802 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:07.423206 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:07.423787 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:07.423790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:07.423792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:07.423884 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:07.424217 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:07.424219 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:07.424221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:07.424458 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:07.424541 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:07.425171 (+ 630us) server_negotiation.cc:299] Negotiation successful
0812 19:04:07.425333 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 19:04:07.426362 1616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:07.412151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.25.130.1:33067)
0812 19:04:07.412816 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:07.412866 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:07.414250 (+ 1384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:07.414862 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:07.414871 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:07.415287 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:07.415822 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:07.415837 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:07.416715 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:07.416719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:07.418087 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:07.418103 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:07.418702 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:07.418706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:07.418947 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:07.419640 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:07.419664 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:07.421546 (+ 1882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:07.423337 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:07.423344 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:07.423361 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:07.423671 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:07.423957 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:07.423959 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:07.423961 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:07.424130 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:07.424565 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:07.424574 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:07.424882 (+ 308us) client_negotiation.cc:769] Sending connection context
0812 19:04:07.425061 (+ 179us) client_negotiation.cc:240] Negotiation successful
0812 19:04:07.425363 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":167,"threads_started":1}
I20260812 19:04:07.427757 1613 heartbeater.cc:347] Connected to a master server at 127.25.130.62:37435
I20260812 19:04:07.428043 1613 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:07.428691 1613 heartbeater.cc:511] Master 127.25.130.62:37435 requested a full tablet report, sending...
I20260812 19:04:07.430683 1376 ts_manager.cc:194] Registered new tserver with Master: 7aa537f1f40c4f53abe5e8e8adec26dc (127.25.130.1:37323)
I20260812 19:04:07.432394 1376 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.130.1@KRBTEST.COM'} at 127.25.130.1:33067
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.2.keytab.
Entry for principal oryx/127.25.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.2.keytab.
I20260812 19:04:07.441874 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:0
--local_ip_for_outbound_sockets=127.25.130.2
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:37435
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.2.keytab
--principal=oryx/127.25.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:45129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:04:07.562773 1619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:07.563083 1619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:07.563133 1619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:07.567415 1619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:07.567509 1619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:07.567575 1619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:04:07.573365 1619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:45129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.2.keytab
--principal=oryx/127.25.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:37435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:07.574586 1619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:07.575670 1619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:07.585835 1625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:07.585867 1626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:07.585834 1629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:07.585898 1619 server_base.cc:1034] running on GCE node
I20260812 19:04:07.586622 1619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:07.587338 1619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:07.588546 1619 hybrid_clock.cc:648] HybridClock initialized: now 1786561447588520 us; error 42 us; skew 500 ppm
Aug 12 19:04:07 dist-test-slave-49h3 krb5kdc[1315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561447, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:07.591621 1619 init.cc:377] Logged in from keytab as oryx/127.25.130.2@KRBTEST.COM (short username oryx)
I20260812 19:04:07.592657 1619 webserver.cc:460] Webserver started at http://127.25.130.2:39539/ using document root <none> and password file <none>
I20260812 19:04:07.593248 1619 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:07.593313 1619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:07.593561 1619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:07.595777 1619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/data/instance:
uuid: "b3d81caeda554d0796d3454b7a08ba83"
format_stamp: "Formatted at 2026-08-12 19:04:07 on dist-test-slave-49h3"
server_key: "f316cc473b52b1ca5160c9ac254eee6d"
server_key_iv: "a8384d23510fd22f0e22c2e1520efa0d"
server_key_version: "encryptionkey@0"
I20260812 19:04:07.596369 1619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance:
uuid: "b3d81caeda554d0796d3454b7a08ba83"
format_stamp: "Formatted at 2026-08-12 19:04:07 on dist-test-slave-49h3"
server_key: "f316cc473b52b1ca5160c9ac254eee6d"
server_key_iv: "a8384d23510fd22f0e22c2e1520efa0d"
server_key_version: "encryptionkey@0"
I20260812 19:04:07.600525 1619 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 19:04:07.603214 1636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:07.604578 1619 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:04:07.604707 1619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "b3d81caeda554d0796d3454b7a08ba83"
format_stamp: "Formatted at 2026-08-12 19:04:07 on dist-test-slave-49h3"
server_key: "f316cc473b52b1ca5160c9ac254eee6d"
server_key_iv: "a8384d23510fd22f0e22c2e1520efa0d"
server_key_version: "encryptionkey@0"
I20260812 19:04:07.604828 1619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:07.617954 1619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:07.621420 1619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:07.621665 1619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:07.622310 1619 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:07.623284 1619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:07.623350 1619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:07.623425 1619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:07.623457 1619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:07.632298 1619 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:41779
I20260812 19:04:07.632330 1782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:41779 every 8 connection(s)
I20260812 19:04:07.633266 1619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:04:07.640070 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 1619
I20260812 19:04:07.640173 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance
I20260812 19:04:07.640422 26120 external_mini_cluster.cc:1428] Setting key d93ce66d11789be07b4ae3860f64c447
Aug 12 19:04:07 dist-test-slave-49h3 krb5kdc[1315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561447, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM
I20260812 19:04:07.648272 1423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:07.635249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:59157 (local address 127.25.130.62:37435)
0812 19:04:07.635442 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:07.635447 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:07.636348 (+ 901us) server_negotiation.cc:407] Connection header received
0812 19:04:07.637837 (+ 1489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:07.637841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:07.637910 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:07.638001 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:07.639218 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:07.639741 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:07.641058 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:07.641338 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:07.641476 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:07.644447 (+ 2971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:07.644451 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:07.644454 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:07.644489 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:07.645909 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:07.646473 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:07.646476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:07.646477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:07.646568 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:07.646919 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:07.646923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:07.646924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:07.647136 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:07.647231 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:07.647926 (+ 695us) server_negotiation.cc:299] Negotiation successful
0812 19:04:07.648087 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 19:04:07.649250 1785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:07.635667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.25.130.2:59157)
0812 19:04:07.636187 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:07.636236 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:07.637606 (+ 1370us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:07.638177 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:07.638185 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:07.638575 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:07.639081 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:07.639094 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:07.639839 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:07.639841 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:07.640892 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:07.640905 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:07.641548 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:07.641551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:07.641735 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:07.642387 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:07.642411 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:07.644229 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:07.646035 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:07.646041 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:07.646056 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:07.646355 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:07.646659 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:07.646662 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:07.646664 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:07.646819 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:07.647247 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:07.647255 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:07.647584 (+ 329us) client_negotiation.cc:769] Sending connection context
0812 19:04:07.647842 (+ 258us) client_negotiation.cc:240] Negotiation successful
0812 19:04:07.648206 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":127,"threads_started":1}
I20260812 19:04:07.650528 1783 heartbeater.cc:347] Connected to a master server at 127.25.130.62:37435
I20260812 19:04:07.650848 1783 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:07.651553 1783 heartbeater.cc:511] Master 127.25.130.62:37435 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.25.130.3@KRBTEST.COM; defaulting to no policy
I20260812 19:04:07.652835 1376 ts_manager.cc:194] Registered new tserver with Master: b3d81caeda554d0796d3454b7a08ba83 (127.25.130.2:41779)
I20260812 19:04:07.653641 1376 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.130.2@KRBTEST.COM'} at 127.25.130.2:59157
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.3.keytab.
Entry for principal oryx/127.25.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.3.keytab.
I20260812 19:04:07.672223 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:0
--local_ip_for_outbound_sockets=127.25.130.3
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:37435
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.3.keytab
--principal=oryx/127.25.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:45129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:04:07.790977 1788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:07.791293 1788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:07.791358 1788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:07.795642 1788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:07.795734 1788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:07.795818 1788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:04:07.801524 1788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:45129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.3.keytab
--principal=oryx/127.25.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:37435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:07.802738 1788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:07.803845 1788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:07.814339 1795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:07.814340 1797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:07.814499 1788 server_base.cc:1034] running on GCE node
W20260812 19:04:07.814721 1801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:07.815201 1788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:07.815934 1788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:07.817111 1788 hybrid_clock.cc:648] HybridClock initialized: now 1786561447817083 us; error 40 us; skew 500 ppm
Aug 12 19:04:07 dist-test-slave-49h3 krb5kdc[1315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561447, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:07.820056 1788 init.cc:377] Logged in from keytab as oryx/127.25.130.3@KRBTEST.COM (short username oryx)
I20260812 19:04:07.821147 1788 webserver.cc:460] Webserver started at http://127.25.130.3:37633/ using document root <none> and password file <none>
I20260812 19:04:07.821805 1788 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:07.821885 1788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:07.822122 1788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:07.824483 1788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/data/instance:
uuid: "867a225140b94ed584a79744322953b7"
format_stamp: "Formatted at 2026-08-12 19:04:07 on dist-test-slave-49h3"
server_key: "ec332c83bfefc7878df555eebcb5412c"
server_key_iv: "e304a4cb3ce229d0c359800da38f7118"
server_key_version: "encryptionkey@0"
I20260812 19:04:07.825111 1788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance:
uuid: "867a225140b94ed584a79744322953b7"
format_stamp: "Formatted at 2026-08-12 19:04:07 on dist-test-slave-49h3"
server_key: "ec332c83bfefc7878df555eebcb5412c"
server_key_iv: "e304a4cb3ce229d0c359800da38f7118"
server_key_version: "encryptionkey@0"
I20260812 19:04:07.829376 1788 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260812 19:04:07.832230 1813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:07.833685 1788 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:04:07.833829 1788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "867a225140b94ed584a79744322953b7"
format_stamp: "Formatted at 2026-08-12 19:04:07 on dist-test-slave-49h3"
server_key: "ec332c83bfefc7878df555eebcb5412c"
server_key_iv: "e304a4cb3ce229d0c359800da38f7118"
server_key_version: "encryptionkey@0"
I20260812 19:04:07.833976 1788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:07.851522 1788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:07.854837 1788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:07.855051 1788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:07.855716 1788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:07.856652 1788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:07.856704 1788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:07.856768 1788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:07.856796 1788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:07.865741 1788 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:40061
I20260812 19:04:07.865774 1957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:40061 every 8 connection(s)
I20260812 19:04:07.866714 1788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:04:07.869853 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 1788
I20260812 19:04:07.869992 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance
I20260812 19:04:07.870270 26120 external_mini_cluster.cc:1428] Setting key c61906a995c5edada7df7fc4969f6b06
Aug 12 19:04:07 dist-test-slave-49h3 krb5kdc[1315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561447, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM
I20260812 19:04:07.880065 1423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:07.868614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:48189 (local address 127.25.130.62:37435)
0812 19:04:07.868762 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:07.868766 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:07.869698 (+ 932us) server_negotiation.cc:407] Connection header received
0812 19:04:07.871165 (+ 1467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:07.871168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:07.871226 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:07.871299 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:07.872531 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:07.873051 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:07.874232 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:07.874445 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:07.874530 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:07.876840 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:07.876843 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:07.876845 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:07.876870 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:07.878020 (+ 1150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:07.878506 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:07.878509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:07.878510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:07.878590 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:07.878919 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:07.878922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:07.878924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:07.879137 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:07.879227 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:07.879749 (+ 522us) server_negotiation.cc:299] Negotiation successful
0812 19:04:07.879890 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 19:04:07.880913 1960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:07.869012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.25.130.3:48189)
0812 19:04:07.869534 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:07.869582 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:07.870943 (+ 1361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:07.871497 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:07.871505 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:07.871881 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:07.872394 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:07.872408 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:07.873173 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:07.873176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:07.874118 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:07.874128 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:07.874583 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:07.874596 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:07.874749 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:07.875261 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:07.875278 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:07.876678 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:07.878130 (+ 1452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:07.878136 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:07.878153 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:07.878408 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:07.878672 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:07.878676 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:07.878678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:07.878829 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:07.879241 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:07.879249 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:07.879505 (+ 256us) client_negotiation.cc:769] Sending connection context
0812 19:04:07.879690 (+ 185us) client_negotiation.cc:240] Negotiation successful
0812 19:04:07.880035 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":130,"threads_started":1}
I20260812 19:04:07.882082 1958 heartbeater.cc:347] Connected to a master server at 127.25.130.62:37435
I20260812 19:04:07.882350 1958 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:07.882961 1958 heartbeater.cc:511] Master 127.25.130.62:37435 requested a full tablet report, sending...
I20260812 19:04:07.884232 1376 ts_manager.cc:194] Registered new tserver with Master: 867a225140b94ed584a79744322953b7 (127.25.130.3:40061)
I20260812 19:04:07.884793 1376 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.130.3@KRBTEST.COM'} at 127.25.130.3:48189
I20260812 19:04:07.896095 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.25.130.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.130.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.130.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.61.keytab.
Entry for principal oryx/127.25.130.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.61.keytab.
I20260812 19:04:07.924244 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.61:40369
--webserver_interface=127.25.130.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.61.keytab
--principal=oryx/127.25.130.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:45129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:37435,127.25.130.61:40369
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:04:08.044216 1964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:08.044538 1964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:08.044582 1964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:08.048816 1964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:04:08.048918 1964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:08.048949 1964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:08.048975 1964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:04:08.048995 1964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:04:08.054741 1964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:45129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:37435,127.25.130.61:40369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.61.keytab
--principal=oryx/127.25.130.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.61:40369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.25.130.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
W20260812 19:04:08.054996 1964 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.25.130.62:37435,127.25.130.61:40369'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 19:04:08.055925 1964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:08.056984 1964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:04:08.065891 1964 server_base.cc:1034] running on GCE node
W20260812 19:04:08.065850 1975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:08.065846 1976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:08.065860 1978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:08.066745 1964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:08.067960 1964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:08.069140 1964 hybrid_clock.cc:648] HybridClock initialized: now 1786561448069121 us; error 36 us; skew 500 ppm
Aug 12 19:04:08 dist-test-slave-49h3 krb5kdc[1315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561448, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:08.072037 1964 init.cc:377] Logged in from keytab as oryx/127.25.130.61@KRBTEST.COM (short username oryx)
I20260812 19:04:08.073060 1964 webserver.cc:460] Webserver started at http://127.25.130.61:36851/ using document root <none> and password file <none>
I20260812 19:04:08.073694 1964 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:08.073776 1964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:08.074003 1964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:08.076236 1964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-1/data/instance:
uuid: "501fbfc27f1746b39f928f6228d3aa5a"
format_stamp: "Formatted at 2026-08-12 19:04:08 on dist-test-slave-49h3"
server_key: "7e42cd2484c00fa42310cf25b010698d"
server_key_iv: "a5808c14e587eced61c55174cd61f781"
server_key_version: "encryptionkey@0"
I20260812 19:04:08.076859 1964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-1/wal/instance:
uuid: "501fbfc27f1746b39f928f6228d3aa5a"
format_stamp: "Formatted at 2026-08-12 19:04:08 on dist-test-slave-49h3"
server_key: "7e42cd2484c00fa42310cf25b010698d"
server_key_iv: "a5808c14e587eced61c55174cd61f781"
server_key_version: "encryptionkey@0"
I20260812 19:04:08.080997 1964 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 19:04:08.083683 1989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:08.085000 1964 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:04:08.085137 1964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-1/wal
uuid: "501fbfc27f1746b39f928f6228d3aa5a"
format_stamp: "Formatted at 2026-08-12 19:04:08 on dist-test-slave-49h3"
server_key: "7e42cd2484c00fa42310cf25b010698d"
server_key_iv: "a5808c14e587eced61c55174cd61f781"
server_key_version: "encryptionkey@0"
I20260812 19:04:08.085247 1964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:08.103266 1964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:08.106277 1964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:08.106467 1964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:08.114595 1964 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.61:40369
I20260812 19:04:08.114858 2051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.61:40369 every 8 connection(s)
I20260812 19:04:08.115695 1964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-1/data/info.pb
I20260812 19:04:08.118649 2052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:08.121831 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 1964
I20260812 19:04:08.122871 2052 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 19:04:08 dist-test-slave-49h3 krb5kdc[1315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561448, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.61@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM
Aug 12 19:04:08 dist-test-slave-49h3 krb5kdc[1315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.130.61@KRBTEST.COM
I20260812 19:04:08.135438 1423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.124887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38626 (local address 127.25.130.62:37435)
0812 19:04:08.125039 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:08.125044 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:08.125485 (+ 441us) server_negotiation.cc:407] Connection header received
0812 19:04:08.125702 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:08.125706 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:08.125760 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:08.125809 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:08.126794 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.127565 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.128590 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.128819 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.128912 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.131549 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:08.131552 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:08.131554 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:08.131584 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:08.133056 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.133679 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.133682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.133684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.133782 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.134098 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.134102 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.134104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.134443 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:08.134538 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:08.135113 (+ 575us) server_negotiation.cc:299] Negotiation successful
0812 19:04:08.135262 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 19:04:08.136363 2058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.124813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.0.0.1:38626)
0812 19:04:08.125336 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:08.125387 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:08.125577 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:08.125975 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:08.125979 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:08.126365 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:08.126647 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.126659 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.127689 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.127691 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.128457 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.128466 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.128969 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.128973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.129124 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.129752 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:08.129775 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:08.131381 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:08.133166 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.133177 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.133190 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.133575 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.133865 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.133867 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.133868 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.134004 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.134539 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:08.134546 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:08.134874 (+ 328us) client_negotiation.cc:769] Sending connection context
0812 19:04:08.135025 (+ 151us) client_negotiation.cc:240] Negotiation successful
0812 19:04:08.135375 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":3,"spinlock_wait_cycles":39296,"thread_start_us":271,"threads_started":1}
I20260812 19:04:08.136365 2057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.123462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54652 (local address 127.25.130.61:40369)
0812 19:04:08.123986 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:08.124005 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:08.124050 (+ 45us) server_negotiation.cc:407] Connection header received
0812 19:04:08.124816 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:08.124839 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:08.125230 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:08.125552 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:08.126419 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.127788 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.128619 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.129006 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.129130 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.131995 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:08.132004 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:08.132021 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:08.132048 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:08.133690 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.134122 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.134126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.134127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.134193 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.134454 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.134456 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.134457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.134854 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:08.135046 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:08.135281 (+ 235us) server_negotiation.cc:299] Negotiation successful
0812 19:04:08.135421 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":364,"spinlock_wait_cycles":2944,"thread_start_us":198,"threads_started":1}
I20260812 19:04:08.137874 2052 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 19:04:08 dist-test-slave-49h3 krb5kdc[1315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561448, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.61@KRBTEST.COM for oryx/127.25.130.61@KRBTEST.COM
I20260812 19:04:08.145401 2058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.138395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:40369 (local address 127.0.0.1:54658)
0812 19:04:08.138515 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:08.138547 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:08.138629 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:08.138926 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:08.138929 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:08.139098 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:08.139189 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.139195 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.139876 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.139879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.140399 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.140406 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.140810 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.140814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.140925 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.141343 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:08.141354 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:08.142644 (+ 1290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:08.144006 (+ 1362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.144009 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.144011 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.144189 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.144425 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.144428 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.144430 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.144502 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.144868 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:08.144871 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:08.144953 (+ 82us) client_negotiation.cc:769] Sending connection context
0812 19:04:08.145036 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 19:04:08.145189 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":5}
I20260812 19:04:08.145401 2057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.138665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54658 (local address 127.25.130.61:40369)
0812 19:04:08.138769 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:08.138772 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:08.138782 (+ 10us) server_negotiation.cc:407] Connection header received
0812 19:04:08.138807 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:08.138809 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:08.138841 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:08.138902 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:08.139290 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.139783 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.140498 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.140720 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.140803 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.142777 (+ 1974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:08.142780 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:08.142782 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:08.142803 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:08.143907 (+ 1104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.144274 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.144278 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.144279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.144349 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.144578 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.144581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.144583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.144763 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:08.144849 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:08.145092 (+ 243us) server_negotiation.cc:299] Negotiation successful
0812 19:04:08.145248 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":80,"server-negotiator.queue_time_us":46}
I20260812 19:04:08.149087 2052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a: Bootstrap starting.
I20260812 19:04:08.151573 2052 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:08.152288 2052 log.cc:824] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:08.154256 2052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a: No bootstrap required, opened a new log
I20260812 19:04:08.157210 2052 raft_consensus.cc:348] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } }
I20260812 19:04:08.157466 2052 raft_consensus.cc:374] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:08.157527 2052 raft_consensus.cc:729] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 501fbfc27f1746b39f928f6228d3aa5a, State: Initialized, Role: FOLLOWER
I20260812 19:04:08.157948 2052 consensus_queue.cc:260] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } }
I20260812 19:04:08.158423 2061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } } }
I20260812 19:04:08.158550 2061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:04:08.159171 2052 sys_catalog.cc:564] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [sys.catalog]: configured and running, proceeding with master startup.
W20260812 19:04:08.164047 2079 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:04:08.164170 2079 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:04:08.165943 1964 master_runner.cc:428] Detected that this master 501fbfc27f1746b39f928f6228d3aa5a is joining an existing cluster
I20260812 19:04:08.166028 1964 master_runner.cc:432] Initiating AddMaster RPC to add 127.25.130.61:40369
I20260812 19:04:08.177675 2087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.168604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.0.0.1:38630)
0812 19:04:08.169118 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:08.169130 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:08.169236 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:08.169570 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:08.169573 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:08.169777 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:08.169916 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.169922 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.171885 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.171888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.172622 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.172635 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.173072 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.173076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.173164 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.173686 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:08.173703 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:08.174335 (+ 632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:08.176071 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.176074 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.176076 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.176363 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.176624 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.176627 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.176629 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.176715 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.177087 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:08.177090 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:08.177195 (+ 105us) client_negotiation.cc:769] Sending connection context
0812 19:04:08.177283 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 19:04:08.177441 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":296,"spinlock_wait_cycles":2816,"thread_start_us":94,"threads_started":1}
I20260812 19:04:08.177659 1423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.168703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38630 (local address 127.25.130.62:37435)
0812 19:04:08.168837 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:08.168842 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:08.169212 (+ 370us) server_negotiation.cc:407] Connection header received
0812 19:04:08.169372 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:08.169376 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:08.169434 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:08.169546 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:08.170977 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.171778 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.172743 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.172969 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.173057 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.174505 (+ 1448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:08.174509 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:08.174512 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:08.174540 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:08.175963 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.176455 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.176459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.176461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.176543 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.176796 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.176800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.176802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.176999 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:08.177087 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:08.177352 (+ 265us) server_negotiation.cc:299] Negotiation successful
0812 19:04:08.177498 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 19:04:08.177982 2057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.168756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54666 (local address 127.25.130.61:40369)
0812 19:04:08.168860 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:08.168865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:08.169091 (+ 226us) server_negotiation.cc:407] Connection header received
0812 19:04:08.169292 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:08.169295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:08.169337 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:08.169404 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:08.170035 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.170817 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.171617 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.171833 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.171924 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.174485 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:08.174489 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:08.174492 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:08.174519 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:08.176369 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.176798 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.176802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.176804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.176873 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.177105 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.177108 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.177110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.177314 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:08.177379 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:08.177717 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 19:04:08.177825 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39,"spinlock_wait_cycles":4480}
I20260812 19:04:08.178306 2086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.168604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:40369 (local address 127.0.0.1:54666)
0812 19:04:08.169012 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:08.169027 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:08.169180 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:08.169415 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:08.169417 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:08.169717 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:08.169906 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.169915 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.170927 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.170930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.171512 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.171520 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.172066 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.172070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.172161 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.173771 (+ 1610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:08.173783 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:08.174340 (+ 557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:08.176479 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.176482 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.176484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.176710 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.176958 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.176961 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.176963 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.177030 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.177432 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:08.177434 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:08.177566 (+ 132us) client_negotiation.cc:769] Sending connection context
0812 19:04:08.178024 (+ 458us) client_negotiation.cc:240] Negotiation successful
0812 19:04:08.178152 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":17,"thread_start_us":103,"threads_started":1}
Aug 12 19:04:08 dist-test-slave-49h3 krb5kdc[1315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561447, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.62@KRBTEST.COM for oryx/127.25.130.61@KRBTEST.COM
I20260812 19:04:08.194296 2057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.183313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54678 (local address 127.25.130.61:40369)
0812 19:04:08.183459 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:08.183464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:08.183866 (+ 402us) server_negotiation.cc:407] Connection header received
0812 19:04:08.184103 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:08.184105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:08.184157 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:08.184230 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:08.185100 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.185729 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.187057 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.187289 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.187374 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.190555 (+ 3181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:08.190560 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:08.190563 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:08.190592 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:08.192145 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.192747 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.192751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.192753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.192842 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.193144 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.193147 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.193149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.193378 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:08.193491 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:08.193977 (+ 486us) server_negotiation.cc:299] Negotiation successful
0812 19:04:08.194111 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 19:04:08.194967 2088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.183269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:40369 (local address 127.0.0.1:54678)
0812 19:04:08.183716 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:08.183777 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:08.183930 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:08.184284 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:08.184290 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:08.184791 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:08.184952 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.184960 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.185841 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.185846 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.186901 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.186914 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.187401 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.187404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.187546 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.188324 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:08.188340 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:08.190406 (+ 2066us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:08.192285 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.192298 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.192306 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.192631 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.192949 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.192952 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.192954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.193041 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.193580 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:08.193587 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:08.193739 (+ 152us) client_negotiation.cc:769] Sending connection context
0812 19:04:08.194526 (+ 787us) client_negotiation.cc:240] Negotiation successful
0812 19:04:08.194783 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":118,"threads_started":1}
I20260812 19:04:08.196918 1376 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.25.130.61:40369: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a"
member_type: NON_VOTER
last_known_addr {
host: "127.25.130.61"
port: 40369
}
attrs {
promote: true
}
}
dest_uuid: "dede9ac8627f4a27b237459ecc7d8ba3"
cas_config_opid_index: -1
I20260812 19:04:08.197815 1376 consensus_queue.cc:237] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: NON_VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: true } }
WARNING: no policy specified for oryx/127.25.130.60@KRBTEST.COM; defaulting to no policy
I20260812 19:04:08.200167 2093 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 501fbfc27f1746b39f928f6228d3aa5a (127.25.130.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: NON_VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: true } } }
I20260812 19:04:08.200695 2021 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:08.202469 2093 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.25.130.61:40369
I20260812 19:04:08.202737 2021 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 1 FOLLOWER]: Refusing update from remote peer dede9ac8627f4a27b237459ecc7d8ba3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 19:04:08.203081 1964 master.cc:537] Master@127.25.130.61:40369 shutting down...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.130.60@KRBTEST.COM" created.
I20260812 19:04:08.203572 2093 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: NON_VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:04:08.204010 2092 sys_catalog.cc:455] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: NON_VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: true } } }
I20260812 19:04:08.204357 2092 sys_catalog.cc:458] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [sys.catalog]: This master's current role is: LEADER
W20260812 19:04:08.205952 1354 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260812 19:04:08.207031 1354 consensus_peers.cc:487] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 -> Peer 501fbfc27f1746b39f928f6228d3aa5a (127.25.130.61:40369): Couldn't send request to peer 501fbfc27f1746b39f928f6228d3aa5a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:04:08.209920 1964 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:04:08.210251 1964 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:04:08.210368 1964 tablet_replica.cc:330] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.130.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.60.keytab.
Entry for principal oryx/127.25.130.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.60.keytab.
I20260812 19:04:08.222465 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.60:44715
--webserver_interface=127.25.130.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.60.keytab
--principal=oryx/127.25.130.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:45129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:37435,127.25.130.61:40369,127.25.130.60:44715
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:04:08.226334 1964 master.cc:552] Master@127.25.130.61:40369 shutdown complete.
I20260812 19:04:08.226410 1964 master_runner.cc:305] Clearing existing system tablet
I20260812 19:04:08.226666 1964 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:08.226723 1964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:08.228108 1964 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 19:04:08.229079 2100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:08.229350 1964 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:08.229509 1964 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 19:04:08.229553 1964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-1/wal
uuid: "501fbfc27f1746b39f928f6228d3aa5a"
format_stamp: "Formatted at 2026-08-12 19:04:08 on dist-test-slave-49h3"
server_key: "7e42cd2484c00fa42310cf25b010698d"
server_key_iv: "a5808c14e587eced61c55174cd61f781"
server_key_version: "encryptionkey@0"
I20260812 19:04:08.231858 1964 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:04:08.234563 1964 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 19:04:08.234681 1964 log.cc:1196] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 19:04:08.234946 1964 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a: Deleting consensus metadata
I20260812 19:04:08.235124 1964 master_runner.cc:315] Copying system tablet from 127.25.130.62:37435
I20260812 19:04:08.236662 1964 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a: tablet copy: Beginning tablet copy session from remote peer at address 127.25.130.62:37435
I20260812 19:04:08.246412 1423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.237592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38646 (local address 127.25.130.62:37435)
0812 19:04:08.237738 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:08.237743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:08.238008 (+ 265us) server_negotiation.cc:407] Connection header received
0812 19:04:08.238158 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:08.238161 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:08.238224 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:08.238315 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:08.238896 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.239740 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.240959 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.241308 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.241421 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.242779 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:08.242782 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:08.242785 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:08.242817 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:08.244462 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.244990 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.244994 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.244996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.245100 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.245384 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.245388 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.245389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.245676 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:08.245787 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:08.246109 (+ 322us) server_negotiation.cc:299] Negotiation successful
0812 19:04:08.246246 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 19:04:08.246449 2107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.237502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.0.0.1:38646)
0812 19:04:08.237887 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:08.237903 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:08.238039 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:08.238347 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:08.238350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:08.238577 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:08.238723 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.238731 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.239864 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.239867 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.240831 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.240843 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.241434 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.241438 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.241574 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.242161 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:08.242175 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:08.242605 (+ 430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:08.244592 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.244595 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.244598 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.244884 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.245190 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.245193 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.245195 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.245293 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.245792 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:08.245795 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:08.245912 (+ 117us) client_negotiation.cc:769] Sending connection context
0812 19:04:08.246027 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 19:04:08.246223 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":99,"threads_started":1}
I20260812 19:04:08.247535 1401 tablet_copy_service.cc:140] P dede9ac8627f4a27b237459ecc7d8ba3: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 501fbfc27f1746b39f928f6228d3aa5a ({username='oryx', principal='oryx/127.25.130.61@KRBTEST.COM'} at 127.0.0.1:38646)
I20260812 19:04:08.247751 1401 tablet_copy_service.cc:161] P dede9ac8627f4a27b237459ecc7d8ba3: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 501fbfc27f1746b39f928f6228d3aa5a at {username='oryx', principal='oryx/127.25.130.61@KRBTEST.COM'} at 127.0.0.1:38646: session id = 501fbfc27f1746b39f928f6228d3aa5a-00000000000000000000000000000000
I20260812 19:04:08.249847 1401 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:04:08.251946 1964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 19:04:08.254706 1964 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a: tablet copy: Starting download of 0 data blocks...
I20260812 19:04:08.254937 1964 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a: tablet copy: Starting download of 1 WAL segments...
I20260812 19:04:08.257083 1964 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:04:08.259434 1401 tablet_copy_service.cc:342] P dede9ac8627f4a27b237459ecc7d8ba3: Request end of tablet copy session 501fbfc27f1746b39f928f6228d3aa5a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.25.130.61@KRBTEST.COM'} at 127.0.0.1:38646
I20260812 19:04:08.259605 1401 tablet_copy_service.cc:433] P dede9ac8627f4a27b237459ecc7d8ba3: ending tablet copy session 501fbfc27f1746b39f928f6228d3aa5a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 501fbfc27f1746b39f928f6228d3aa5a
I20260812 19:04:08.268952 1964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:08.270790 2108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:08.270872 2112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:08.271473 2109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:08.271807 1964 server_base.cc:1034] running on GCE node
I20260812 19:04:08.271981 1964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:08.272307 1964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:08.273480 1964 hybrid_clock.cc:648] HybridClock initialized: now 1786561448273458 us; error 37 us; skew 500 ppm
Aug 12 19:04:08 dist-test-slave-49h3 krb5kdc[1315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561448, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:08.275096 1964 init.cc:377] Logged in from keytab as oryx/127.25.130.61@KRBTEST.COM (short username oryx)
I20260812 19:04:08.275617 1964 webserver.cc:460] Webserver started at http://127.25.130.61:37543/ using document root <none> and password file <none>
I20260812 19:04:08.275822 1964 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:08.275866 1964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:08.277289 1964 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 19:04:08.278398 2120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:08.278657 1964 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 19:04:08.278789 1964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-1/wal
uuid: "501fbfc27f1746b39f928f6228d3aa5a"
format_stamp: "Formatted at 2026-08-12 19:04:08 on dist-test-slave-49h3"
server_key: "7e42cd2484c00fa42310cf25b010698d"
server_key_iv: "a5808c14e587eced61c55174cd61f781"
server_key_version: "encryptionkey@0"
I20260812 19:04:08.278949 1964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:08.287601 1964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:08.290117 1964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:08.295217 1964 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.61:40369
I20260812 19:04:08.295257 2184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.61:40369 every 8 connection(s)
I20260812 19:04:08.296036 1964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-1/data/info.pb
I20260812 19:04:08.298303 2186 sys_catalog.cc:263] Verifying existing consensus state
I20260812 19:04:08.299464 2186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a: Bootstrap starting.
I20260812 19:04:08.311555 2186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:04:08.311898 2186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a: Bootstrap complete.
I20260812 19:04:08.312579 2186 raft_consensus.cc:348] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: NON_VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: true } }
I20260812 19:04:08.312745 2186 raft_consensus.cc:729] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 501fbfc27f1746b39f928f6228d3aa5a, State: Initialized, Role: LEARNER
I20260812 19:04:08.312858 2186 consensus_queue.cc:260] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: NON_VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: true } }
I20260812 19:04:08.313386 2186 sys_catalog.cc:564] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:04:08.313315 2190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: NON_VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: true } } }
I20260812 19:04:08.313660 2190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [sys.catalog]: This master's current role is: LEARNER
I20260812 19:04:08.317217 2202 catalog_manager.cc:1259] Loaded cluster ID: a674a8bb57f4493194ed80aa887c9dca
I20260812 19:04:08.317301 2202 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a: loading cluster ID for follower catalog manager: success
I20260812 19:04:08.319664 2202 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a: acquiring CA information for follower catalog manager: success
I20260812 19:04:08.321283 2202 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 19:04:08.362151 2098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:08.362471 2098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:08.362533 2098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:08.366746 2098 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:04:08.366840 2098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:08.366869 2098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:08.366894 2098 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:04:08.366917 2098 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:04:08.372700 2098 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:45129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:37435,127.25.130.61:40369,127.25.130.60:44715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.60.keytab
--principal=oryx/127.25.130.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.60:44715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.25.130.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:08.373973 2098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:08.375059 2098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:08.384016 2208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:08.383996 2207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:08.384001 2211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:08.384446 2098 server_base.cc:1034] running on GCE node
I20260812 19:04:08.385114 2098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:08.386368 2098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:08.387555 2098 hybrid_clock.cc:648] HybridClock initialized: now 1786561448387528 us; error 39 us; skew 500 ppm
Aug 12 19:04:08 dist-test-slave-49h3 krb5kdc[1315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561448, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:08.390518 2098 init.cc:377] Logged in from keytab as oryx/127.25.130.60@KRBTEST.COM (short username oryx)
I20260812 19:04:08.391510 2098 webserver.cc:460] Webserver started at http://127.25.130.60:34267/ using document root <none> and password file <none>
I20260812 19:04:08.392103 2098 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:08.392176 2098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:08.392387 2098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:08.394701 2098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-2/data/instance:
uuid: "5ef909f4f0644d49ad5990de41b6e21c"
format_stamp: "Formatted at 2026-08-12 19:04:08 on dist-test-slave-49h3"
server_key: "e84ea264add642dd913e8502dca337d0"
server_key_iv: "dc3814497bae494daaf008320eda955f"
server_key_version: "encryptionkey@0"
I20260812 19:04:08.395303 2098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-2/wal/instance:
uuid: "5ef909f4f0644d49ad5990de41b6e21c"
format_stamp: "Formatted at 2026-08-12 19:04:08 on dist-test-slave-49h3"
server_key: "e84ea264add642dd913e8502dca337d0"
server_key_iv: "dc3814497bae494daaf008320eda955f"
server_key_version: "encryptionkey@0"
I20260812 19:04:08.399396 2098 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:04:08.402164 2218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:08.403555 2098 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:04:08.403716 2098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-2/wal
uuid: "5ef909f4f0644d49ad5990de41b6e21c"
format_stamp: "Formatted at 2026-08-12 19:04:08 on dist-test-slave-49h3"
server_key: "e84ea264add642dd913e8502dca337d0"
server_key_iv: "dc3814497bae494daaf008320eda955f"
server_key_version: "encryptionkey@0"
I20260812 19:04:08.403839 2098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:08.435631 1613 heartbeater.cc:503] Master 127.25.130.62:37435 was elected leader, sending a full tablet report...
I20260812 19:04:08.440001 2098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:08.443041 2098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:08.443238 2098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:08.451167 2098 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.60:44715
I20260812 19:04:08.451476 2276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.60:44715 every 8 connection(s)
I20260812 19:04:08.452263 2098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-2/data/info.pb
I20260812 19:04:08.454068 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 2098
I20260812 19:04:08.455529 2277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:08.459568 2277 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 19:04:08 dist-test-slave-49h3 krb5kdc[1315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.130.60@KRBTEST.COM
Aug 12 19:04:08 dist-test-slave-49h3 krb5kdc[1315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561448, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.60@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM
I20260812 19:04:08.468904 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.455625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34584 (local address 127.25.130.60:44715)
0812 19:04:08.456101 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:08.456120 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:08.456159 (+ 39us) server_negotiation.cc:407] Connection header received
0812 19:04:08.456846 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:08.456863 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:08.457229 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:08.457590 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:08.458434 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.459469 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.460384 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.460863 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.461010 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.463861 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:08.463866 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:08.463875 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:08.463903 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:08.465731 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.466181 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.466193 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.466198 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.466316 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.466583 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.466586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.466587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.467000 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:08.467181 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:08.467506 (+ 325us) server_negotiation.cc:299] Negotiation successful
0812 19:04:08.467832 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":169,"threads_started":1}
I20260812 19:04:08.470155 1423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.461500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38660 (local address 127.25.130.62:37435)
0812 19:04:08.461633 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:08.461637 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:08.461827 (+ 190us) server_negotiation.cc:407] Connection header received
0812 19:04:08.461957 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:08.461960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:08.462016 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:08.462109 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:08.462861 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.463425 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.464385 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.464613 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.464707 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.466872 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:08.466875 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:08.466877 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:08.466904 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:08.468365 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.468803 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.468807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.468808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.468913 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.469235 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.469238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.469240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.469434 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:08.469568 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:08.469881 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 19:04:08.469998 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 19:04:08.470183 2281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.461273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.0.0.1:38660)
0812 19:04:08.461688 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:08.461740 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:08.461862 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:08.462191 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:08.462197 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:08.462578 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:08.462727 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.462735 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.463586 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.463589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.464258 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.464267 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.464719 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.464722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.464839 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.465261 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:08.465272 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:08.466726 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:08.468483 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.468491 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.468496 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.468710 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.469004 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.469007 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.469009 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.469141 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.469563 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:08.469569 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:08.469685 (+ 116us) client_negotiation.cc:769] Sending connection context
0812 19:04:08.469807 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 19:04:08.469999 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":109,"threads_started":1}
I20260812 19:04:08.471258 2277 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 19:04:08 dist-test-slave-49h3 krb5kdc[1315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561448, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.60@KRBTEST.COM for oryx/127.25.130.61@KRBTEST.COM
I20260812 19:04:08.479278 2281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.471746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:40369 (local address 127.0.0.1:54694)
0812 19:04:08.471867 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:08.471878 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:08.471941 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:08.472351 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:08.472354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:08.472513 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:08.472609 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.472614 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.473520 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.473524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.474066 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.474075 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.474509 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.474512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.474610 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.475053 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:08.475064 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:08.476277 (+ 1213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:08.477835 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.477839 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.477841 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.478040 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.478298 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.478301 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.478303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.478377 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.478752 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:08.478755 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:08.478847 (+ 92us) client_negotiation.cc:769] Sending connection context
0812 19:04:08.478931 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 19:04:08.479093 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 19:04:08.479279 2282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.471872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54694 (local address 127.25.130.61:40369)
0812 19:04:08.472151 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:08.472155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:08.472165 (+ 10us) server_negotiation.cc:407] Connection header received
0812 19:04:08.472202 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:08.472204 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:08.472241 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:08.472317 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:08.472722 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.473398 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.474180 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.474408 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.474500 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.476425 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:08.476428 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:08.476430 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:08.476457 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:08.477728 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.478129 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.478132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.478133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.478214 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.478456 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.478459 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.478461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.478660 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:08.478749 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:08.479000 (+ 251us) server_negotiation.cc:299] Negotiation successful
0812 19:04:08.479126 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":109,"threads_started":1}
I20260812 19:04:08.480249 2277 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.60" port: 44715 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 19:04:08 dist-test-slave-49h3 krb5kdc[1315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561448, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.60@KRBTEST.COM for oryx/127.25.130.60@KRBTEST.COM
I20260812 19:04:08.488196 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.480850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34594 (local address 127.25.130.60:44715)
0812 19:04:08.480947 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:08.480952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:08.480964 (+ 12us) server_negotiation.cc:407] Connection header received
0812 19:04:08.480990 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:08.480993 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:08.481028 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:08.481087 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:08.481418 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.481979 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.482682 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.483442 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.483549 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.485405 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:08.485408 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:08.485410 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:08.485432 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:08.486751 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.487121 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.487123 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.487124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.487191 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.487382 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.487384 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.487385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.487561 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:08.487635 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:08.487910 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 19:04:08.488057 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":37}
I20260812 19:04:08.488196 2281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.480730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:44715 (local address 127.0.0.1:34594)
0812 19:04:08.480830 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:08.480842 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:08.480911 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:08.481108 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:08.481111 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:08.481256 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:08.481323 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.481327 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.482100 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.482103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.482584 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.482591 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.483555 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.483559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.483641 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.484051 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:08.484062 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:08.485256 (+ 1194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:08.486858 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.486860 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.486862 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.487051 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.487252 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.487254 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.487255 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.487319 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.487676 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:08.487679 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:08.487772 (+ 93us) client_negotiation.cc:769] Sending connection context
0812 19:04:08.487869 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 19:04:08.487997 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":4}
I20260812 19:04:08.491220 2277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c: Bootstrap starting.
I20260812 19:04:08.493698 2277 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:08.494457 2277 log.cc:824] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:08.496465 2277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c: No bootstrap required, opened a new log
I20260812 19:04:08.499620 2277 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 44715 } }
I20260812 19:04:08.499864 2277 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:08.499917 2277 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ef909f4f0644d49ad5990de41b6e21c, State: Initialized, Role: FOLLOWER
I20260812 19:04:08.500376 2277 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 44715 } }
I20260812 19:04:08.500909 2284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 44715 } } }
I20260812 19:04:08.501051 2284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:04:08.501714 2277 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [sys.catalog]: configured and running, proceeding with master startup.
W20260812 19:04:08.506814 2301 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:04:08.506913 2301 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:04:08.510541 2098 master_runner.cc:428] Detected that this master 5ef909f4f0644d49ad5990de41b6e21c is joining an existing cluster
I20260812 19:04:08.510628 2098 master_runner.cc:432] Initiating AddMaster RPC to add 127.25.130.60:44715
I20260812 19:04:08.522574 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.513877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34610 (local address 127.25.130.60:44715)
0812 19:04:08.514174 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:08.514179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:08.514383 (+ 204us) server_negotiation.cc:407] Connection header received
0812 19:04:08.514509 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:08.514511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:08.514544 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:08.514585 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:08.515283 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.516095 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.517475 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.517792 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.517864 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.519183 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:08.519187 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:08.519190 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:08.519216 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:08.520828 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.521243 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.521246 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.521248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.521338 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.521655 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.521658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.521660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.521894 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:08.521959 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:08.522285 (+ 326us) server_negotiation.cc:299] Negotiation successful
0812 19:04:08.522415 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":230}
I20260812 19:04:08.522585 2307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.513806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:44715 (local address 127.0.0.1:34610)
0812 19:04:08.514316 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:08.514328 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:08.514427 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:08.514709 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:08.514711 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:08.514984 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:08.515163 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.515173 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.516260 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.516263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.517322 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.517334 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.518041 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.518044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.518135 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.518656 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:08.518670 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:08.519036 (+ 366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:08.520945 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.520948 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.520950 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.521153 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.521423 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.521426 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.521428 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.521558 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.522015 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:08.522017 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:08.522105 (+ 88us) client_negotiation.cc:769] Sending connection context
0812 19:04:08.522186 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 19:04:08.522357 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"mutex_wait_us":69,"thread_start_us":117,"threads_started":1}
I20260812 19:04:08.522696 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 1453
I20260812 19:04:08.535036 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:37323
--local_ip_for_outbound_sockets=127.25.130.1
--tserver_master_addrs=127.25.130.62:37435
--webserver_port=37503
--webserver_interface=127.25.130.1
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.1.keytab
--principal=oryx/127.25.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:45129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.25.130.62:37435,127.25.130.61:40369
--tserver_master_addrs=127.25.130.62:37435,127.25.130.61:40369,127.25.130.60:44715 with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:04:08.559084 2282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.514179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54696 (local address 127.25.130.61:40369)
0812 19:04:08.514315 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:08.514319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:08.514333 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:04:08.514379 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:08.514382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:08.514436 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:08.514506 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:08.515311 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.516096 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.517517 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.517870 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.517938 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.555224 (+ 37286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:08.555229 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:08.555231 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:08.555267 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:08.557048 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.557642 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.557646 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.557648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.557745 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.558054 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.558058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.558059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.558314 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:08.558398 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:08.558760 (+ 362us) server_negotiation.cc:299] Negotiation successful
0812 19:04:08.558912 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 19:04:08.559311 1423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.513333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38674 (local address 127.25.130.62:37435)
0812 19:04:08.513498 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:08.513503 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:08.514187 (+ 684us) server_negotiation.cc:407] Connection header received
0812 19:04:08.514425 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:08.514429 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:08.514486 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:08.514538 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:08.515371 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.516189 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.517581 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.517948 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.518027 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.555216 (+ 37189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:08.555222 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:08.555225 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:08.555260 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:08.557393 (+ 2133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.557970 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.557974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.557976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.558079 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.558372 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.558375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.558377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.558596 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:08.558684 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:08.559019 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 19:04:08.559149 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 19:04:08.559803 2309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.513278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:40369 (local address 127.0.0.1:54696)
0812 19:04:08.513784 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:08.513798 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:08.513916 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:08.514628 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:08.514631 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:08.514941 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:08.515164 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.515173 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.516249 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.516252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.517357 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.517364 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.518073 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.518076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.518168 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.554314 (+ 36146us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:08.554333 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:08.554964 (+ 631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:08.557224 (+ 2260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.557227 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.557230 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.557519 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.557862 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.557865 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.557867 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.557954 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.558457 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:08.558459 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:08.558577 (+ 118us) client_negotiation.cc:769] Sending connection context
0812 19:04:08.559351 (+ 774us) client_negotiation.cc:240] Negotiation successful
0812 19:04:08.559524 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":180,"thread_start_us":148,"threads_started":1}
I20260812 19:04:08.559859 2310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.513285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.0.0.1:38674)
0812 19:04:08.514089 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:08.514102 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:08.514301 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:08.514758 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:08.514761 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:08.515046 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:08.515178 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.515186 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.516348 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.516352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.517413 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.517425 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.518196 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.518199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.518296 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.554182 (+ 35886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:08.554204 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:08.554966 (+ 762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:08.557581 (+ 2615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.557585 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.557587 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.557851 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.558187 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.558190 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.558192 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.558277 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.558742 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:08.558745 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:08.558840 (+ 95us) client_negotiation.cc:769] Sending connection context
0812 19:04:08.559515 (+ 675us) client_negotiation.cc:240] Negotiation successful
0812 19:04:08.559679 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"mutex_wait_us":105,"thread_start_us":128,"threads_started":1}
Aug 12 19:04:08 dist-test-slave-49h3 krb5kdc[1315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561447, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.62@KRBTEST.COM for oryx/127.25.130.60@KRBTEST.COM
I20260812 19:04:08.573698 2088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.564136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:44715 (local address 127.0.0.1:34626)
0812 19:04:08.564317 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:08.564332 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:08.564417 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:08.564706 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:08.564709 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:08.564925 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:08.565041 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.565048 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.566044 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.566047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.566915 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.566926 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.567474 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.567478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.567572 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.568098 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:08.568113 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:08.569806 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:08.571830 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.571834 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.571836 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.572105 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.572458 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.572461 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.572463 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.572559 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.573072 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:08.573075 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:08.573202 (+ 127us) client_negotiation.cc:769] Sending connection context
0812 19:04:08.573305 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 19:04:08.573515 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260812 19:04:08.574242 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.564227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34626 (local address 127.25.130.60:44715)
0812 19:04:08.564340 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:08.564344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:08.564376 (+ 32us) server_negotiation.cc:407] Connection header received
0812 19:04:08.564539 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:08.564542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:08.564599 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:08.564681 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:08.565156 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.565927 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.567038 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.567360 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.567463 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.570015 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:08.570019 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:08.570022 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:08.570052 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:08.571680 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.572250 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.572254 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.572256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.572348 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.572667 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.572671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.572672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.572922 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:08.573017 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:08.573950 (+ 933us) server_negotiation.cc:299] Negotiation successful
0812 19:04:08.574083 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 19:04:08.575300 1376 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.25.130.60:44715: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c"
member_type: NON_VOTER
last_known_addr {
host: "127.25.130.60"
port: 44715
}
attrs {
promote: true
}
}
dest_uuid: "dede9ac8627f4a27b237459ecc7d8ba3"
cas_config_opid_index: 5
I20260812 19:04:08.575982 1376 consensus_queue.cc:237] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: NON_VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: true } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: NON_VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: true } }
I20260812 19:04:08.577140 2092 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 5ef909f4f0644d49ad5990de41b6e21c (127.25.130.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: NON_VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: true } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: NON_VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: true } } }
I20260812 19:04:08.578421 2250 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:08.578887 2092 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.25.130.60:44715
I20260812 19:04:08.579124 2092 sys_catalog.cc:455] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: NON_VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: true } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: NON_VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: true } } }
I20260812 19:04:08.579242 2092 sys_catalog.cc:458] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:08.579954 2098 master.cc:537] Master@127.25.130.60:44715 shutting down...
I20260812 19:04:08.579958 2250 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [term 1 FOLLOWER]: Refusing update from remote peer dede9ac8627f4a27b237459ecc7d8ba3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 19:04:08.581252 2093 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: NON_VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 19:04:08.583760 1354 consensus_peers.cc:487] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 -> Peer 5ef909f4f0644d49ad5990de41b6e21c (127.25.130.60:44715): Couldn't send request to peer 5ef909f4f0644d49ad5990de41b6e21c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:04:08.586021 2098 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:04:08.586458 2098 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:04:08.586659 2098 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c: stopping tablet replica
I20260812 19:04:08.586964 2282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.577665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54700 (local address 127.25.130.61:40369)
0812 19:04:08.580428 (+ 2763us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:08.580434 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:08.581179 (+ 745us) server_negotiation.cc:407] Connection header received
0812 19:04:08.581648 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:08.581652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:08.581729 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:08.581801 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:08.582351 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.583120 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.585308 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.586034 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.586155 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.586337 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:08.586721 (+ 384us) server_negotiation.cc:299] Negotiation successful
0812 19:04:08.586818 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2656}
I20260812 19:04:08.587381 2088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.577410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:40369 (local address 127.0.0.1:54700)
0812 19:04:08.581063 (+ 3653us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:08.581081 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:08.581350 (+ 269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:08.581984 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:08.581988 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:08.582008 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:08.582213 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.582221 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.583339 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.583341 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.585141 (+ 1800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.585156 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.586416 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.586420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.586526 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.586592 (+ 66us) client_negotiation.cc:769] Sending connection context
0812 19:04:08.587139 (+ 547us) client_negotiation.cc:240] Negotiation successful
0812 19:04:08.587256 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3560}
I20260812 19:04:08.588016 2157 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 1 LEARNER]: Refusing update from remote peer dede9ac8627f4a27b237459ecc7d8ba3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 19:04:08.588480 2093 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: NON_VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:04:08.590641 2157 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 5ef909f4f0644d49ad5990de41b6e21c (127.25.130.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: NON_VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: true } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: NON_VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: true } } }
I20260812 19:04:08.593225 2098 master.cc:552] Master@127.25.130.60:44715 shutdown complete.
I20260812 19:04:08.593330 2098 master_runner.cc:305] Clearing existing system tablet
I20260812 19:04:08.593256 2190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader dede9ac8627f4a27b237459ecc7d8ba3. Latest consensus state: current_term: 1 leader_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: NON_VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: true } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: NON_VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: true } } }
I20260812 19:04:08.593423 2190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [sys.catalog]: This master's current role is: LEARNER
I20260812 19:04:08.593562 2190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: NON_VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: true } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: NON_VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: true } } }
I20260812 19:04:08.593654 2190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [sys.catalog]: This master's current role is: LEARNER
I20260812 19:04:08.593848 2098 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:08.593920 2098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:08.595268 2098 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 19:04:08.596236 2316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:08.596520 2098 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:08.596661 2098 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 19:04:08.596704 2098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-2/wal
uuid: "5ef909f4f0644d49ad5990de41b6e21c"
format_stamp: "Formatted at 2026-08-12 19:04:08 on dist-test-slave-49h3"
server_key: "e84ea264add642dd913e8502dca337d0"
server_key_iv: "dc3814497bae494daaf008320eda955f"
server_key_version: "encryptionkey@0"
I20260812 19:04:08.598948 2098 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:04:08.599105 2093 raft_consensus.cc:1050] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3: attempting to promote NON_VOTER 501fbfc27f1746b39f928f6228d3aa5a to VOTER
I20260812 19:04:08.599633 2093 consensus_queue.cc:237] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: NON_VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: true } }
I20260812 19:04:08.601883 2088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.601360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:44715 (local address 127.0.0.1:34628)
0812 19:04:08.601676 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:08.601787 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:44715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230}
I20260812 19:04:08.601864 2098 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 19:04:08.601970 2098 log.cc:1196] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 19:04:08.602238 2098 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c: Deleting consensus metadata
I20260812 19:04:08.602411 2098 master_runner.cc:315] Copying system tablet from 127.25.130.62:37435
I20260812 19:04:08.602720 2157 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 1 LEARNER]: Refusing update from remote peer dede9ac8627f4a27b237459ecc7d8ba3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 19:04:08.603703 2098 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c: tablet copy: Beginning tablet copy session from remote peer at address 127.25.130.62:37435
I20260812 19:04:08.603832 2092 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 19:04:08.604856 2088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.604483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:44715 (local address 127.0.0.1:34640)
0812 19:04:08.604710 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:08.604770 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:44715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
W20260812 19:04:08.605948 1354 consensus_peers.cc:487] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 -> Peer 5ef909f4f0644d49ad5990de41b6e21c (127.25.130.60:44715): Couldn't send request to peer 5ef909f4f0644d49ad5990de41b6e21c. Status: Network error: Client connection negotiation failed: client connection to 127.25.130.60:44715: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 19:04:08.607147 2092 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 501fbfc27f1746b39f928f6228d3aa5a (127.25.130.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: NON_VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: true } } }
I20260812 19:04:08.608891 2092 sys_catalog.cc:455] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: NON_VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: true } } }
I20260812 19:04:08.609014 2092 sys_catalog.cc:458] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:08.609143 2092 sys_catalog.cc:455] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: NON_VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: true } } }
I20260812 19:04:08.609226 2092 sys_catalog.cc:458] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:08.610551 2157 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 501fbfc27f1746b39f928f6228d3aa5a (127.25.130.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: NON_VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: true } } }
I20260812 19:04:08.613955 2314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: NON_VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: true } } }
I20260812 19:04:08.614123 2314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:04:08.616487 1423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.604384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38688 (local address 127.25.130.62:37435)
0812 19:04:08.604508 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:08.604512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:08.605082 (+ 570us) server_negotiation.cc:407] Connection header received
0812 19:04:08.605277 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:08.605281 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:08.605329 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:08.605409 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:08.606156 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.606978 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.608272 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.608626 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.608706 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.610324 (+ 1618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:08.610328 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:08.610331 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:08.610379 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:08.614279 (+ 3900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.614974 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.614978 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.614980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.615103 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.615475 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.615478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.615480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.615759 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:08.615865 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:08.616167 (+ 302us) server_negotiation.cc:299] Negotiation successful
0812 19:04:08.616319 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 19:04:08.616505 2323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.604605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.0.0.1:38688)
0812 19:04:08.604995 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:08.605007 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:08.605167 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:08.605630 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:08.605644 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:08.605888 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:08.606000 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.606024 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.607130 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.607134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.608135 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.608149 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.608875 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.608878 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.609039 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.609675 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:08.609692 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:08.610149 (+ 457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:08.614519 (+ 4370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.614523 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.614525 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.614850 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.615265 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.615269 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.615271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.615377 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.615865 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:08.615868 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:08.615981 (+ 113us) client_negotiation.cc:769] Sending connection context
0812 19:04:08.616093 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 19:04:08.616266 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":106,"threads_started":1}
I20260812 19:04:08.617009 1401 tablet_copy_service.cc:140] P dede9ac8627f4a27b237459ecc7d8ba3: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5ef909f4f0644d49ad5990de41b6e21c ({username='oryx', principal='oryx/127.25.130.60@KRBTEST.COM'} at 127.0.0.1:38688)
I20260812 19:04:08.617113 1401 tablet_copy_service.cc:161] P dede9ac8627f4a27b237459ecc7d8ba3: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5ef909f4f0644d49ad5990de41b6e21c at {username='oryx', principal='oryx/127.25.130.60@KRBTEST.COM'} at 127.0.0.1:38688: session id = 5ef909f4f0644d49ad5990de41b6e21c-00000000000000000000000000000000
I20260812 19:04:08.618297 1401 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:04:08.619769 2098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 19:04:08.622566 2098 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c: tablet copy: Starting download of 0 data blocks...
I20260812 19:04:08.622787 2098 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c: tablet copy: Starting download of 1 WAL segments...
I20260812 19:04:08.624766 2098 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:04:08.627005 1401 tablet_copy_service.cc:342] P dede9ac8627f4a27b237459ecc7d8ba3: Request end of tablet copy session 5ef909f4f0644d49ad5990de41b6e21c-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.25.130.60@KRBTEST.COM'} at 127.0.0.1:38688
I20260812 19:04:08.627133 1401 tablet_copy_service.cc:433] P dede9ac8627f4a27b237459ecc7d8ba3: ending tablet copy session 5ef909f4f0644d49ad5990de41b6e21c-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5ef909f4f0644d49ad5990de41b6e21c
I20260812 19:04:08.641129 2098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:08.643067 2324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:08.643131 2330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:08.643555 2326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:08.645990 2098 server_base.cc:1034] running on GCE node
I20260812 19:04:08.646227 2098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:08.646612 2098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:08.647791 2098 hybrid_clock.cc:648] HybridClock initialized: now 1786561448647768 us; error 46 us; skew 500 ppm
Aug 12 19:04:08 dist-test-slave-49h3 krb5kdc[1315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561448, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:08.649626 2098 init.cc:377] Logged in from keytab as oryx/127.25.130.60@KRBTEST.COM (short username oryx)
I20260812 19:04:08.650173 2098 webserver.cc:460] Webserver started at http://127.25.130.60:39303/ using document root <none> and password file <none>
I20260812 19:04:08.650390 2098 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:08.650436 2098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:08.651932 2098 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 19:04:08.653084 2337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:08.653363 2098 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 19:04:08.653555 2098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-2/wal
uuid: "5ef909f4f0644d49ad5990de41b6e21c"
format_stamp: "Formatted at 2026-08-12 19:04:08 on dist-test-slave-49h3"
server_key: "e84ea264add642dd913e8502dca337d0"
server_key_iv: "dc3814497bae494daaf008320eda955f"
server_key_version: "encryptionkey@0"
I20260812 19:04:08.653656 2098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:08.656037 1783 heartbeater.cc:503] Master 127.25.130.62:37435 was elected leader, sending a full tablet report...
I20260812 19:04:08.672163 2098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:08.674436 2098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:08.679988 2098 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.60:44715
I20260812 19:04:08.680791 2396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.60:44715 every 8 connection(s)
I20260812 19:04:08.680841 2098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/master-2/data/info.pb
I20260812 19:04:08.682943 2397 sys_catalog.cc:263] Verifying existing consensus state
I20260812 19:04:08.683985 2397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c: Bootstrap starting.
W20260812 19:04:08.688290 2311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:08.688568 2311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:08.688633 2311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:08.692960 2311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:08.693033 2311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:08.693122 2311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:04:08.693894 2397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:04:08.694173 2397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c: Bootstrap complete.
I20260812 19:04:08.694734 2397 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: NON_VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: true } }
I20260812 19:04:08.694857 2397 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5ef909f4f0644d49ad5990de41b6e21c, State: Initialized, Role: LEARNER
I20260812 19:04:08.694949 2397 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: NON_VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: true } }
I20260812 19:04:08.695292 2405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: NON_VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: true } } }
I20260812 19:04:08.695413 2405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [sys.catalog]: This master's current role is: LEARNER
I20260812 19:04:08.695429 2397 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:04:08.698195 2416 catalog_manager.cc:1259] Loaded cluster ID: a674a8bb57f4493194ed80aa887c9dca
I20260812 19:04:08.698251 2416 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c: loading cluster ID for follower catalog manager: success
I20260812 19:04:08.699476 2311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:45129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.1.keytab
--principal=oryx/127.25.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:37323
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=37503
--tserver_master_addrs=127.25.130.62:37435,127.25.130.61:40369,127.25.130.60:44715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:08.699962 2416 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c: acquiring CA information for follower catalog manager: success
I20260812 19:04:08.700613 2311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:08.701121 2416 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 19:04:08.701740 2311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:08.712028 2418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:08.712029 2420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:08.712029 2422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:08.712366 2311 server_base.cc:1034] running on GCE node
I20260812 19:04:08.712898 2311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:08.713665 2311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:08.714841 2311 hybrid_clock.cc:648] HybridClock initialized: now 1786561448714795 us; error 58 us; skew 500 ppm
Aug 12 19:04:08 dist-test-slave-49h3 krb5kdc[1315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561448, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:08.717808 2311 init.cc:377] Logged in from keytab as oryx/127.25.130.1@KRBTEST.COM (short username oryx)
I20260812 19:04:08.718770 2311 webserver.cc:460] Webserver started at http://127.25.130.1:37503/ using document root <none> and password file <none>
I20260812 19:04:08.719329 2311 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:08.719413 2311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:08.723572 2311 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 19:04:08.725847 2431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:08.727221 2311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:04:08.727340 2311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "7aa537f1f40c4f53abe5e8e8adec26dc"
format_stamp: "Formatted at 2026-08-12 19:04:07 on dist-test-slave-49h3"
server_key: "338e00ddb6ae993f06f5d229e39fad3d"
server_key_iv: "fbbe4626e91c9c42efd432358efe6845"
server_key_version: "encryptionkey@0"
I20260812 19:04:08.727916 2311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:08.746587 2311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:08.749922 2311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:08.750119 2311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:08.750735 2311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:08.751816 2311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:08.751883 2311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:08.751943 2311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:08.751965 2311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:08.762125 2311 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:37323
I20260812 19:04:08.762148 2561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:37323 every 8 connection(s)
I20260812 19:04:08.763206 2311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:04:08.763862 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 2311
I20260812 19:04:08.764077 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 1619
Aug 12 19:04:08 dist-test-slave-49h3 krb5kdc[1315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561448, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.61@KRBTEST.COM
Aug 12 19:04:08 dist-test-slave-49h3 krb5kdc[1315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561448, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM
I20260812 19:04:08.784042 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:41779
--local_ip_for_outbound_sockets=127.25.130.2
--tserver_master_addrs=127.25.130.62:37435
--webserver_port=39539
--webserver_interface=127.25.130.2
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.2.keytab
--principal=oryx/127.25.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:45129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.25.130.62:37435,127.25.130.61:40369
--tserver_master_addrs=127.25.130.62:37435,127.25.130.61:40369,127.25.130.60:44715 with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 19:04:08 dist-test-slave-49h3 krb5kdc[1315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561448, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.60@KRBTEST.COM
I20260812 19:04:08.788391 1423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.765144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:59939 (local address 127.25.130.62:37435)
0812 19:04:08.773867 (+ 8723us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:08.773872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:08.773897 (+ 25us) server_negotiation.cc:407] Connection header received
0812 19:04:08.773957 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:08.773961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:08.774020 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:08.774197 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:08.775916 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.776682 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.778486 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.778846 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.778936 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.782951 (+ 4015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:08.782955 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:08.782958 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:08.782991 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:08.785036 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.786280 (+ 1244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.786284 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.786286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.786401 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.786895 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.786898 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.786900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.787146 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:08.787235 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:08.788068 (+ 833us) server_negotiation.cc:299] Negotiation successful
0812 19:04:08.788216 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8585}
I20260812 19:04:08.789565 2568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.775230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:40369 (local address 127.25.130.1:47203)
0812 19:04:08.775792 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:08.775805 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:08.775891 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:08.776421 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:08.776424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:08.776669 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:08.776805 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.776812 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.777973 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.777976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.778851 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.778861 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.779633 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.779636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.779721 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.780167 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:08.780179 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:08.782967 (+ 2788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:08.785121 (+ 2154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.785128 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.785147 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.785950 (+ 803us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.786513 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.786516 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.786518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.786776 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.787295 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:08.787303 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:08.787748 (+ 445us) client_negotiation.cc:769] Sending connection context
0812 19:04:08.787979 (+ 231us) client_negotiation.cc:240] Negotiation successful
0812 19:04:08.788384 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"mutex_wait_us":27,"spinlock_wait_cycles":8192,"thread_start_us":361,"threads_started":1}
I20260812 19:04:08.789901 2567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.774683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:44715 (local address 127.25.130.1:38641)
0812 19:04:08.775103 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:08.775120 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:08.775242 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:08.778572 (+ 3330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:08.778575 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:08.778808 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:08.778936 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.778943 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.780796 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.780799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.781689 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.781701 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.782347 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.782351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.782432 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.783305 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:08.783319 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:08.785002 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:08.787941 (+ 2939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.787944 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.787946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.788192 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.788507 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.788511 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.788513 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.788617 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.789042 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:08.789045 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:08.789137 (+ 92us) client_negotiation.cc:769] Sending connection context
0812 19:04:08.789208 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 19:04:08.789341 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":183,"threads_started":1}
I20260812 19:04:08.789986 2282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.775961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:47203 (local address 127.25.130.61:40369)
0812 19:04:08.776116 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:08.776121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:08.776135 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:04:08.776180 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:08.776183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:08.776231 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:08.776303 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:08.776945 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.777731 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.779007 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.779355 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.779432 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.783110 (+ 3678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:08.783113 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:08.783115 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:08.783143 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:08.784720 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.786297 (+ 1577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.786301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.786303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.786400 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.786886 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.786889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.786891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.787145 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:08.787235 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:08.789701 (+ 2466us) server_negotiation.cc:299] Negotiation successful
0812 19:04:08.789844 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 19:04:08.789577 2566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.765624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.25.130.1:59939)
0812 19:04:08.766536 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:08.766613 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:08.768669 (+ 2056us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:08.774474 (+ 5805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:08.774486 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:08.774990 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:08.775702 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.775724 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.776878 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.776881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.778333 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.778343 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.779171 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.779175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.779412 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.780112 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:08.780135 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:08.782699 (+ 2564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:08.785175 (+ 2476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.785179 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.785181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.786156 (+ 975us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.786513 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.786516 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.786518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.786776 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.787294 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:08.787303 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:08.787747 (+ 444us) client_negotiation.cc:769] Sending connection context
0812 19:04:08.788642 (+ 895us) client_negotiation.cc:240] Negotiation successful
0812 19:04:08.788792 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":51,"spinlock_wait_cycles":15232,"thread_start_us":438,"threads_started":1}
I20260812 19:04:08.790489 2570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.777508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:38641 (local address 127.25.130.60:44715)
0812 19:04:08.778296 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:08.778300 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:08.778315 (+ 15us) server_negotiation.cc:407] Connection header received
0812 19:04:08.778360 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:08.778363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:08.778410 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:08.779058 (+ 648us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:08.779183 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.780665 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.781844 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.782190 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.782679 (+ 489us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.785167 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:08.785171 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:08.785174 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:08.785202 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:08.787823 (+ 2621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.788306 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.788309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.788311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.788403 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.788723 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.788726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.788727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.788934 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:08.790105 (+ 1171us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:08.790185 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 19:04:08.790320 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":708,"thread_start_us":91,"threads_started":1}
I20260812 19:04:08.791285 2563 heartbeater.cc:347] Connected to a master server at 127.25.130.62:37435
I20260812 19:04:08.791325 2564 heartbeater.cc:347] Connected to a master server at 127.25.130.61:40369
I20260812 19:04:08.791471 2562 heartbeater.cc:347] Connected to a master server at 127.25.130.60:44715
I20260812 19:04:08.791568 2563 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:08.791566 2562 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:08.791567 2564 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:08.792303 2563 heartbeater.cc:511] Master 127.25.130.62:37435 requested a full tablet report, sending...
I20260812 19:04:08.792369 2562 heartbeater.cc:511] Master 127.25.130.60:44715 requested a full tablet report, sending...
I20260812 19:04:08.792304 2564 heartbeater.cc:511] Master 127.25.130.61:40369 requested a full tablet report, sending...
I20260812 19:04:08.793709 1376 ts_manager.cc:194] Re-registered known tserver with Master: 7aa537f1f40c4f53abe5e8e8adec26dc (127.25.130.1:37323)
I20260812 19:04:08.794114 2354 ts_manager.cc:194] Registered new tserver with Master: 7aa537f1f40c4f53abe5e8e8adec26dc (127.25.130.1:37323)
I20260812 19:04:08.794359 1376 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.130.1@KRBTEST.COM'} at 127.25.130.1:59939
I20260812 19:04:08.795372 2137 ts_manager.cc:194] Registered new tserver with Master: 7aa537f1f40c4f53abe5e8e8adec26dc (127.25.130.1:37323)
I20260812 19:04:08.887025 1958 heartbeater.cc:503] Master 127.25.130.62:37435 was elected leader, sending a full tablet report...
W20260812 19:04:08.909125 2571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:08.909426 2571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:08.909505 2571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:08.913852 2571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:08.913936 2571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:08.914031 2571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:04:08.919822 2571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:45129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.2.keytab
--principal=oryx/127.25.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:41779
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=39539
--tserver_master_addrs=127.25.130.62:37435,127.25.130.61:40369,127.25.130.60:44715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:08.920992 2571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:08.922139 2571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:08.932287 2580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:08.932303 2578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:08.932287 2577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:08.932642 2571 server_base.cc:1034] running on GCE node
I20260812 19:04:08.933235 2571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:08.933982 2571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:08.935166 2571 hybrid_clock.cc:648] HybridClock initialized: now 1786561448935142 us; error 41 us; skew 500 ppm
Aug 12 19:04:08 dist-test-slave-49h3 krb5kdc[1315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561448, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:08.938182 2571 init.cc:377] Logged in from keytab as oryx/127.25.130.2@KRBTEST.COM (short username oryx)
I20260812 19:04:08.939221 2571 webserver.cc:460] Webserver started at http://127.25.130.2:39539/ using document root <none> and password file <none>
I20260812 19:04:08.939777 2571 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:08.939859 2571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:08.943966 2571 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 19:04:08.946229 2589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:08.947541 2571 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:04:08.947661 2571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "b3d81caeda554d0796d3454b7a08ba83"
format_stamp: "Formatted at 2026-08-12 19:04:07 on dist-test-slave-49h3"
server_key: "f316cc473b52b1ca5160c9ac254eee6d"
server_key_iv: "a8384d23510fd22f0e22c2e1520efa0d"
server_key_version: "encryptionkey@0"
I20260812 19:04:08.948274 2571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:08.960381 2571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:08.963690 2571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:08.963886 2571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:08.964569 2571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:08.965543 2571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:08.965592 2571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:08.965659 2571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:08.965689 2571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:08.976018 2571 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:41779
I20260812 19:04:08.976043 2722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:41779 every 8 connection(s)
I20260812 19:04:08.977185 2571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:04:08.985002 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 2571
I20260812 19:04:08.985206 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 1788
Aug 12 19:04:08 dist-test-slave-49h3 krb5kdc[1315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561448, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.60@KRBTEST.COM
Aug 12 19:04:08 dist-test-slave-49h3 krb5kdc[1315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561448, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.61@KRBTEST.COM
I20260812 19:04:08.994745 2570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.979336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:59197 (local address 127.25.130.60:44715)
0812 19:04:08.979475 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:08.979478 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:08.981013 (+ 1535us) server_negotiation.cc:407] Connection header received
0812 19:04:08.982142 (+ 1129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:08.982146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:08.982209 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:08.982261 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:08.983749 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.984593 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.986501 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.986852 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.986949 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.989951 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:08.989957 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:08.989959 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:08.989991 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:08.991912 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.992688 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.992692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.992694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.992789 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.993214 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.993218 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.993220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.993497 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:08.993582 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:08.994421 (+ 839us) server_negotiation.cc:299] Negotiation successful
0812 19:04:08.994569 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 19:04:08.994745 2282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.981474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:57987 (local address 127.25.130.61:40369)
0812 19:04:08.981604 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:08.981609 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:08.981626 (+ 17us) server_negotiation.cc:407] Connection header received
0812 19:04:08.982098 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:08.982101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:08.982158 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:08.982238 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:08.984766 (+ 2528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.985344 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.986403 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.986716 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.986823 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.990178 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:08.990182 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:08.990185 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:08.990219 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:08.992358 (+ 2139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.992842 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.992846 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.992848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.992943 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:08.993239 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:08.993242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:08.993243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:08.993492 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:08.993582 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:08.994421 (+ 839us) server_negotiation.cc:299] Negotiation successful
0812 19:04:08.994569 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 19:04:08.995041 2088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.990171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:44715 (local address 127.0.0.1:34652)
0812 19:04:08.990471 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:08.990485 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:08.990580 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:08.991134 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:08.991138 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:08.991162 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:08.991281 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.991288 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.992423 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.992427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.993664 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.993679 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.994616 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.994620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.994726 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.994757 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 19:04:08.994812 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 19:04:08.994901 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212}
I20260812 19:04:08.996032 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:40061
--local_ip_for_outbound_sockets=127.25.130.3
--tserver_master_addrs=127.25.130.62:37435
--webserver_port=37633
--webserver_interface=127.25.130.3
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.3.keytab
--principal=oryx/127.25.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:45129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.25.130.62:37435,127.25.130.61:40369
--tserver_master_addrs=127.25.130.62:37435,127.25.130.61:40369,127.25.130.60:44715 with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:04:08.996625 2732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.990473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34652 (local address 127.25.130.60:44715)
0812 19:04:08.990858 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:08.990863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:08.990875 (+ 12us) server_negotiation.cc:407] Connection header received
0812 19:04:08.990919 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:08.990921 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:08.990980 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:08.991402 (+ 422us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:08.991505 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.992282 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.993823 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.994444 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.995902 (+ 1458us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.996086 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:08.996362 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 19:04:08.996448 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":158,"threads_started":1}
I20260812 19:04:08.996659 2730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.979756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:44715 (local address 127.25.130.2:59197)
0812 19:04:08.980904 (+ 1148us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:08.980916 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:08.981880 (+ 964us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:08.982436 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:08.982444 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:08.982951 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:08.983591 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.983598 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.984782 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.984786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.986299 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.986313 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.987054 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.987057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.987163 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.987864 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:08.987884 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:08.989685 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:08.992061 (+ 2376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.992069 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.992092 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.992554 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.992892 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.992895 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.992897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.993103 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.993657 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:08.993665 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:08.994103 (+ 438us) client_negotiation.cc:769] Sending connection context
0812 19:04:08.994991 (+ 888us) client_negotiation.cc:240] Negotiation successful
0812 19:04:08.995458 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1055,"mutex_wait_us":190,"spinlock_wait_cycles":82048,"thread_start_us":207,"threads_started":1}
I20260812 19:04:08.996655 2731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.981062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:40369 (local address 127.25.130.2:57987)
0812 19:04:08.981368 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:08.981383 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:08.981881 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:08.982434 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:08.982442 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:08.982935 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:08.983577 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.983592 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.985496 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.985500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.986274 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.986285 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.986897 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.986902 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:08.987114 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:08.987901 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:08.987915 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:08.989993 (+ 2078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:08.992499 (+ 2506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.992503 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.992505 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.992731 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.993049 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:08.993052 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:08.993054 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:08.993141 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:08.993657 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:08.993665 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:08.994103 (+ 438us) client_negotiation.cc:769] Sending connection context
0812 19:04:08.994991 (+ 888us) client_negotiation.cc:240] Negotiation successful
0812 19:04:08.995523 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":245,"spinlock_wait_cycles":14208,"thread_start_us":107,"threads_started":1}
I20260812 19:04:08.997967 2724 heartbeater.cc:347] Connected to a master server at 127.25.130.60:44715
I20260812 19:04:08.998041 2727 heartbeater.cc:347] Connected to a master server at 127.25.130.61:40369
I20260812 19:04:08.998245 2724 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:08.998243 2727 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:08.998309 2365 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260812 19:04:08.998740 2405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [sys.catalog]: SysCatalogTable state changed. Reason: New leader dede9ac8627f4a27b237459ecc7d8ba3. Latest consensus state: current_term: 1 leader_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: NON_VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: true } } }
I20260812 19:04:08.998889 2405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [sys.catalog]: This master's current role is: LEARNER
I20260812 19:04:08.998965 2727 heartbeater.cc:511] Master 127.25.130.61:40369 requested a full tablet report, sending...
I20260812 19:04:08.998965 2724 heartbeater.cc:511] Master 127.25.130.60:44715 requested a full tablet report, sending...
I20260812 19:04:09.000229 2354 ts_manager.cc:194] Registered new tserver with Master: b3d81caeda554d0796d3454b7a08ba83 (127.25.130.2:41779)
I20260812 19:04:09.000262 2137 ts_manager.cc:194] Registered new tserver with Master: b3d81caeda554d0796d3454b7a08ba83 (127.25.130.2:41779)
Aug 12 19:04:09 dist-test-slave-49h3 krb5kdc[1315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561448, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM
I20260812 19:04:09.007167 1423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.979238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:43203 (local address 127.25.130.62:37435)
0812 19:04:08.979411 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:08.979416 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:08.980864 (+ 1448us) server_negotiation.cc:407] Connection header received
0812 19:04:08.982153 (+ 1289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:08.982157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:08.982209 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:08.982261 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:08.983771 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.984593 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.000343 (+ 15750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.000649 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.000758 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:09.003749 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:09.003753 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:09.003755 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:09.003786 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:09.005203 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:09.005944 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:09.005947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:09.005948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:09.006042 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:09.006326 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:09.006328 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:09.006329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:09.006517 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:09.006590 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:09.006903 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 19:04:09.007016 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 19:04:09.007366 2729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:08.979756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.25.130.2:43203)
0812 19:04:08.980673 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:08.980737 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:08.981880 (+ 1143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:08.982436 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:08.982444 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:08.982936 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:08.983587 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:08.983597 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:08.999362 (+ 15765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:08.999366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:09.000185 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:09.000198 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.000791 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.000795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:09.000888 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:09.001983 (+ 1095us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:09.001999 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:09.003556 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:09.005333 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:09.005337 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:09.005339 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:09.005843 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:09.006130 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:09.006133 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:09.006135 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:09.006239 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:09.006604 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:09.006607 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:09.006744 (+ 137us) client_negotiation.cc:769] Sending connection context
0812 19:04:09.006854 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 19:04:09.007047 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"mutex_wait_us":101,"spinlock_wait_cycles":27264,"thread_start_us":717,"threads_started":1}
I20260812 19:04:09.008039 2726 heartbeater.cc:347] Connected to a master server at 127.25.130.62:37435
I20260812 19:04:09.008167 2726 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:09.008374 2726 heartbeater.cc:511] Master 127.25.130.62:37435 requested a full tablet report, sending...
I20260812 19:04:09.009141 1376 ts_manager.cc:194] Re-registered known tserver with Master: b3d81caeda554d0796d3454b7a08ba83 (127.25.130.2:41779)
I20260812 19:04:09.009971 1376 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.130.2@KRBTEST.COM'} at 127.25.130.2:43203
W20260812 19:04:09.117589 2733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:09.117877 2733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:09.117923 2733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:09.122020 2733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:09.122090 2733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:09.122154 2733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:04:09.127609 2733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:45129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.3.keytab
--principal=oryx/127.25.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:40061
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=37633
--tserver_master_addrs=127.25.130.62:37435,127.25.130.61:40369,127.25.130.60:44715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:09.128852 2733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:09.129907 2733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:09.139685 2740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:09.139685 2739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:09.139714 2742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:09.139753 2733 server_base.cc:1034] running on GCE node
I20260812 19:04:09.140424 2733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:09.141070 2733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:09.142226 2733 hybrid_clock.cc:648] HybridClock initialized: now 1786561449142199 us; error 40 us; skew 500 ppm
Aug 12 19:04:09 dist-test-slave-49h3 krb5kdc[1315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561449, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:09.144977 2733 init.cc:377] Logged in from keytab as oryx/127.25.130.3@KRBTEST.COM (short username oryx)
I20260812 19:04:09.145998 2733 webserver.cc:460] Webserver started at http://127.25.130.3:37633/ using document root <none> and password file <none>
I20260812 19:04:09.146579 2733 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:09.146663 2733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:09.150899 2733 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 19:04:09.153263 2751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:09.154675 2733 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:04:09.154798 2733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "867a225140b94ed584a79744322953b7"
format_stamp: "Formatted at 2026-08-12 19:04:07 on dist-test-slave-49h3"
server_key: "ec332c83bfefc7878df555eebcb5412c"
server_key_iv: "e304a4cb3ce229d0c359800da38f7118"
server_key_version: "encryptionkey@0"
I20260812 19:04:09.155375 2733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:09.181795 2733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:09.184753 2733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:09.184952 2733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:09.185612 2733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:09.186558 2733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:09.186609 2733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:09.186666 2733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:09.186697 2733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:09.195489 2733 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:40061
I20260812 19:04:09.195547 2886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:40061 every 8 connection(s)
I20260812 19:04:09.196617 2733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:04:09.206415 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 2733
Aug 12 19:04:09 dist-test-slave-49h3 krb5kdc[1315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561449, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.60@KRBTEST.COM
Aug 12 19:04:09 dist-test-slave-49h3 krb5kdc[1315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561449, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.61@KRBTEST.COM
Aug 12 19:04:09 dist-test-slave-49h3 krb5kdc[1315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561449, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM
I20260812 19:04:09.213714 1423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:09.198524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:53565 (local address 127.25.130.62:37435)
0812 19:04:09.198674 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:09.198677 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:09.199732 (+ 1055us) server_negotiation.cc:407] Connection header received
0812 19:04:09.200871 (+ 1139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:09.200875 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:09.200931 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:09.200983 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:09.202457 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.203246 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.204378 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.204695 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.204796 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:09.208950 (+ 4154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:09.208954 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:09.208957 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:09.208987 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:09.210704 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:09.211972 (+ 1268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:09.211976 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:09.211978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:09.212074 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:09.212384 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:09.212387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:09.212389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:09.212609 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:09.212692 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:09.213385 (+ 693us) server_negotiation.cc:299] Negotiation successful
0812 19:04:09.213560 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 19:04:09.214300 2893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:09.199010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:40369 (local address 127.25.130.3:40381)
0812 19:04:09.199947 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:09.199960 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:09.200644 (+ 684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:09.201172 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:09.201180 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:09.201762 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:09.202284 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:09.202293 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.203125 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.203127 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:09.204372 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:09.204381 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.205034 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.205037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:09.205113 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:09.205743 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:09.205766 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:09.208258 (+ 2492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:09.210316 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:09.210324 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:09.210343 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:09.210742 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:09.211094 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:09.211097 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:09.211099 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:09.211304 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:09.211930 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:09.211938 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:09.212309 (+ 371us) client_negotiation.cc:769] Sending connection context
0812 19:04:09.212549 (+ 240us) client_negotiation.cc:240] Negotiation successful
0812 19:04:09.212971 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":847,"mutex_wait_us":73,"spinlock_wait_cycles":53120,"thread_start_us":181,"threads_started":1}
I20260812 19:04:09.214920 2892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:09.199010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.25.130.3:53565)
0812 19:04:09.199577 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:09.199621 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:09.200644 (+ 1023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:09.201172 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:09.201180 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:09.201760 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:09.202286 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:09.202293 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.203362 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.203365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:09.204256 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:09.204266 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.204841 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.204845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:09.205042 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:09.205789 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:09.205801 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:09.208766 (+ 2965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:09.210833 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:09.210835 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:09.210837 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:09.211064 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:09.212177 (+ 1113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:09.212180 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:09.212182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:09.212279 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:09.212735 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:09.212738 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:09.212828 (+ 90us) client_negotiation.cc:769] Sending connection context
0812 19:04:09.212928 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 19:04:09.213043 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"mutex_wait_us":144,"spinlock_wait_cycles":33792,"thread_start_us":679,"threads_started":1}
I20260812 19:04:09.215849 2282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:09.198769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:40381 (local address 127.25.130.61:40369)
0812 19:04:09.198885 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:09.198888 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:09.200034 (+ 1146us) server_negotiation.cc:407] Connection header received
0812 19:04:09.200871 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:09.200875 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:09.200931 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:09.200983 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:09.202446 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.202976 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.204496 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.204825 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.204917 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:09.208451 (+ 3534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:09.208455 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:09.208458 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:09.208485 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:09.210209 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:09.210869 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:09.210873 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:09.210874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:09.210965 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:09.211498 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:09.211502 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:09.211504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:09.211771 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:09.211868 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:09.215542 (+ 3674us) server_negotiation.cc:299] Negotiation successful
0812 19:04:09.215678 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 19:04:09.216066 2889 heartbeater.cc:347] Connected to a master server at 127.25.130.62:37435
I20260812 19:04:09.216326 2889 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:09.216943 2889 heartbeater.cc:511] Master 127.25.130.62:37435 requested a full tablet report, sending...
I20260812 19:04:09.217002 2732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:09.200334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:44255 (local address 127.25.130.60:44715)
0812 19:04:09.200459 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:09.200464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:09.200539 (+ 75us) server_negotiation.cc:407] Connection header received
0812 19:04:09.200848 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:09.200852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:09.200906 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:09.200981 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:09.202419 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.203246 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.204664 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.204949 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.205048 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:09.208699 (+ 3651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:09.208703 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:09.208706 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:09.208735 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:09.211851 (+ 3116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:09.215542 (+ 3691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:09.215546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:09.215548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:09.215658 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:09.215983 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:09.215986 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:09.215988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:09.216259 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:09.216341 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:09.216708 (+ 367us) server_negotiation.cc:299] Negotiation successful
0812 19:04:09.216845 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 19:04:09.217521 2894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:09.200206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:44715 (local address 127.25.130.3:44255)
0812 19:04:09.200470 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:09.200481 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:09.200643 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:09.201170 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:09.201178 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:09.201719 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:09.202274 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:09.202288 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.203749 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.203752 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:09.204558 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:09.204565 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.205207 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.205210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:09.205328 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:09.205911 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:09.205924 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:09.208534 (+ 2610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:09.215112 (+ 6578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:09.215116 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:09.215118 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:09.215408 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:09.215784 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:09.215787 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:09.215789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:09.215876 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:09.216420 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:09.216423 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:09.216528 (+ 105us) client_negotiation.cc:769] Sending connection context
0812 19:04:09.217210 (+ 682us) client_negotiation.cc:240] Negotiation successful
0812 19:04:09.217358 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":21,"thread_start_us":77,"threads_started":1}
I20260812 19:04:09.217557 2890 heartbeater.cc:347] Connected to a master server at 127.25.130.61:40369
I20260812 19:04:09.217638 2890 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:09.217804 2890 heartbeater.cc:511] Master 127.25.130.61:40369 requested a full tablet report, sending...
I20260812 19:04:09.218269 2887 heartbeater.cc:347] Connected to a master server at 127.25.130.60:44715
I20260812 19:04:09.218351 2887 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:09.218509 2887 heartbeater.cc:511] Master 127.25.130.60:44715 requested a full tablet report, sending...
I20260812 19:04:09.218775 1376 ts_manager.cc:194] Re-registered known tserver with Master: 867a225140b94ed584a79744322953b7 (127.25.130.3:40061)
I20260812 19:04:09.218990 2137 ts_manager.cc:194] Registered new tserver with Master: 867a225140b94ed584a79744322953b7 (127.25.130.3:40061)
I20260812 19:04:09.219199 2354 ts_manager.cc:194] Registered new tserver with Master: 867a225140b94ed584a79744322953b7 (127.25.130.3:40061)
I20260812 19:04:09.219511 1376 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.130.3@KRBTEST.COM'} at 127.25.130.3:53565
I20260812 19:04:09.220275 2902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:09.208442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54716 (local address 127.25.130.61:40369)
0812 19:04:09.209973 (+ 1531us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:09.209978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:09.210098 (+ 120us) server_negotiation.cc:407] Connection header received
0812 19:04:09.210249 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:09.210252 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:09.210301 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:09.210379 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:09.211431 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.212215 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.213515 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.213865 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.213957 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:09.215907 (+ 1950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:09.215911 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:09.215914 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:09.215941 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:09.217682 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:09.218413 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:09.218416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:09.218418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:09.218510 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:09.218816 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:09.218819 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:09.218821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:09.219521 (+ 700us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:09.219660 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:09.219988 (+ 328us) server_negotiation.cc:299] Negotiation successful
0812 19:04:09.220118 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1449,"thread_start_us":79,"threads_started":1}
I20260812 19:04:09.224969 2901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:09.208348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38694 (local address 127.25.130.62:37435)
0812 19:04:09.209812 (+ 1464us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:09.209816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:09.211427 (+ 1611us) server_negotiation.cc:407] Connection header received
0812 19:04:09.211592 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:09.211594 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:09.211646 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:09.211769 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:09.212738 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.213546 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.214670 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.218298 (+ 3628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.218375 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:09.221820 (+ 3445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:09.221823 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:09.221826 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:09.221854 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:09.223365 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:09.223779 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:09.223783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:09.223784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:09.223866 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:09.224131 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:09.224134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:09.224136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:09.224338 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:09.224446 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:09.224684 (+ 238us) server_negotiation.cc:299] Negotiation successful
0812 19:04:09.224792 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1375,"thread_start_us":101,"threads_started":1}
I20260812 19:04:09.228027 2570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:09.210718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34660 (local address 127.25.130.60:44715)
0812 19:04:09.210838 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:09.210843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:09.220450 (+ 9607us) server_negotiation.cc:407] Connection header received
0812 19:04:09.220653 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:09.220656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:09.220703 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:09.220767 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:09.221741 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.222483 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.223269 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.223582 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.223659 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:09.224882 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:09.224886 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:09.224889 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:09.224914 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:09.226451 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:09.226883 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:09.226886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:09.226888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:09.226985 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:09.227235 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:09.227238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:09.227239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:09.227411 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:09.227506 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:09.227755 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 19:04:09.227879 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 19:04:09.228080 1376 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38694:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:04:09.231494 1376 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:04:09.234272 2317 sys_catalog.cc:455] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: NON_VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: true } } }
I20260812 19:04:09.234429 2317 sys_catalog.cc:458] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:09.238039 2317 raft_consensus.cc:1050] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3: attempting to promote NON_VOTER 5ef909f4f0644d49ad5990de41b6e21c to VOTER
I20260812 19:04:09.238374 2317 consensus_queue.cc:237] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: false } }
I20260812 19:04:09.241321 2364 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [term 1 LEARNER]: Refusing update from remote peer dede9ac8627f4a27b237459ecc7d8ba3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 19:04:09.241564 2157 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 1 FOLLOWER]: Refusing update from remote peer dede9ac8627f4a27b237459ecc7d8ba3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 19:04:09.241961 2908 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260812 19:04:09.242826 2317 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 19:04:09.244081 2908 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 5ef909f4f0644d49ad5990de41b6e21c (127.25.130.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: false } } }
I20260812 19:04:09.244524 2157 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 5ef909f4f0644d49ad5990de41b6e21c (127.25.130.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: false } } }
I20260812 19:04:09.245173 2364 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 5ef909f4f0644d49ad5990de41b6e21c (127.25.130.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: false } } }
I20260812 19:04:09.245810 2908 sys_catalog.cc:455] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: false } } }
I20260812 19:04:09.245910 2908 sys_catalog.cc:458] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:09.246058 2908 sys_catalog.cc:455] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: false } } }
I20260812 19:04:09.246129 2908 sys_catalog.cc:458] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:09.246130 2317 sys_catalog.cc:455] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: false } } }
I20260812 19:04:09.246304 2317 sys_catalog.cc:458] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3 [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:09.246659 2919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: false } } }
I20260812 19:04:09.246788 2919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:04:09.247133 2405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: false } } }
I20260812 19:04:09.247225 2405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:04:09.254058 2088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:09.247997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:40061 (local address 127.0.0.1:57764)
0812 19:04:09.248184 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:09.248209 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:09.248285 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:09.249832 (+ 1547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:09.249835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:09.249853 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:09.249967 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:09.249973 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.251008 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.251011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:09.252348 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:09.252358 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.253654 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.253657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:09.253742 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:09.253766 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 19:04:09.253829 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 19:04:09.253907 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":250}
I20260812 19:04:09.254161 2921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:09.248154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57764 (local address 127.25.130.3:40061)
0812 19:04:09.249101 (+ 947us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:09.249125 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:09.249153 (+ 28us) server_negotiation.cc:407] Connection header received
0812 19:04:09.249239 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:09.249245 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:09.249688 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:09.249863 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:09.250062 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.250914 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.252482 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.253542 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.253642 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:09.253883 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:09.253938 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 19:04:09.254035 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":838,"thread_start_us":121,"threads_started":1}
I20260812 19:04:09.254271 2922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:09.248325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:41779 (local address 127.0.0.1:37022)
0812 19:04:09.248585 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:09.248598 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:09.248700 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:09.250066 (+ 1366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:09.250068 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:09.250086 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:09.250193 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:09.250199 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.251731 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.251734 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:09.252876 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:09.252888 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.253876 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.253880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:09.253974 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:09.253999 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 19:04:09.254045 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 19:04:09.254116 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":22,"spinlock_wait_cycles":9600,"thread_start_us":118,"threads_started":1}
I20260812 19:04:09.255264 2925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:09.248845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:37323 (local address 127.0.0.1:46088)
0812 19:04:09.249086 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:09.249098 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:09.249246 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:09.250086 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:09.250088 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:09.250108 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:09.250205 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:09.250211 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.251557 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.251560 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:09.253369 (+ 1809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:09.253379 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.254875 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.254879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:09.254962 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:09.254983 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 19:04:09.255044 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 19:04:09.255127 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":22,"thread_start_us":110,"threads_started":1}
I20260812 19:04:09.255558 2923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:09.248506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37022 (local address 127.25.130.2:41779)
0812 19:04:09.249645 (+ 1139us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:09.249657 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:09.249681 (+ 24us) server_negotiation.cc:407] Connection header received
0812 19:04:09.249754 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:09.249758 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:09.249939 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:09.250090 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:09.250288 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.251636 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.253008 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.253735 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.254926 (+ 1191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:09.255079 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:09.255131 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 19:04:09.255229 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1020,"thread_start_us":95,"threads_started":1}
I20260812 19:04:09.255775 2926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:09.249034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46088 (local address 127.25.130.1:37323)
0812 19:04:09.249388 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:09.249405 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:09.249428 (+ 23us) server_negotiation.cc:407] Connection header received
0812 19:04:09.249529 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:09.249534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:09.249694 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:09.249863 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:09.250336 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.251432 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.253543 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.254527 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.254634 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:09.254780 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:09.255515 (+ 735us) server_negotiation.cc:299] Negotiation successful
0812 19:04:09.255627 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":76,"threads_started":1}
I20260812 19:04:09.257364 2814 tablet_service.cc:1467] Processing CreateTablet for tablet bc8f42d404ed4094ab116c0df6995b82 (DEFAULT_TABLE table=test-table [id=c07c28a7f7df466ba3c0a2b690c5952f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:04:09.258400 2814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc8f42d404ed4094ab116c0df6995b82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:09.258538 2488 tablet_service.cc:1467] Processing CreateTablet for tablet bc8f42d404ed4094ab116c0df6995b82 (DEFAULT_TABLE table=test-table [id=c07c28a7f7df466ba3c0a2b690c5952f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:04:09.258531 2648 tablet_service.cc:1467] Processing CreateTablet for tablet bc8f42d404ed4094ab116c0df6995b82 (DEFAULT_TABLE table=test-table [id=c07c28a7f7df466ba3c0a2b690c5952f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:04:09.259614 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc8f42d404ed4094ab116c0df6995b82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:09.259624 2488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc8f42d404ed4094ab116c0df6995b82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:09.265954 2927 tablet_bootstrap.cc:492] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7: Bootstrap starting.
I20260812 19:04:09.267789 2929 tablet_bootstrap.cc:492] T bc8f42d404ed4094ab116c0df6995b82 P b3d81caeda554d0796d3454b7a08ba83: Bootstrap starting.
I20260812 19:04:09.268486 2927 tablet_bootstrap.cc:654] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:09.269481 2927 log.cc:824] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:09.269824 2928 tablet_bootstrap.cc:492] T bc8f42d404ed4094ab116c0df6995b82 P 7aa537f1f40c4f53abe5e8e8adec26dc: Bootstrap starting.
I20260812 19:04:09.270179 2929 tablet_bootstrap.cc:654] T bc8f42d404ed4094ab116c0df6995b82 P b3d81caeda554d0796d3454b7a08ba83: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:09.271057 2929 log.cc:824] T bc8f42d404ed4094ab116c0df6995b82 P b3d81caeda554d0796d3454b7a08ba83: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:09.271781 2927 tablet_bootstrap.cc:492] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7: No bootstrap required, opened a new log
I20260812 19:04:09.271862 2928 tablet_bootstrap.cc:654] T bc8f42d404ed4094ab116c0df6995b82 P 7aa537f1f40c4f53abe5e8e8adec26dc: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:09.271993 2927 ts_tablet_manager.cc:1397] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 19:04:09.272614 2928 log.cc:824] T bc8f42d404ed4094ab116c0df6995b82 P 7aa537f1f40c4f53abe5e8e8adec26dc: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:09.273380 2929 tablet_bootstrap.cc:492] T bc8f42d404ed4094ab116c0df6995b82 P b3d81caeda554d0796d3454b7a08ba83: No bootstrap required, opened a new log
I20260812 19:04:09.273622 2929 ts_tablet_manager.cc:1397] T bc8f42d404ed4094ab116c0df6995b82 P b3d81caeda554d0796d3454b7a08ba83: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 19:04:09.274406 2928 tablet_bootstrap.cc:492] T bc8f42d404ed4094ab116c0df6995b82 P 7aa537f1f40c4f53abe5e8e8adec26dc: No bootstrap required, opened a new log
I20260812 19:04:09.274585 2928 ts_tablet_manager.cc:1397] T bc8f42d404ed4094ab116c0df6995b82 P 7aa537f1f40c4f53abe5e8e8adec26dc: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 19:04:09.276223 2927 raft_consensus.cc:348] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d81caeda554d0796d3454b7a08ba83" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41779 } } peers { permanent_uuid: "867a225140b94ed584a79744322953b7" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40061 } } peers { permanent_uuid: "7aa537f1f40c4f53abe5e8e8adec26dc" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37323 } }
I20260812 19:04:09.276566 2927 raft_consensus.cc:374] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:09.276640 2927 raft_consensus.cc:729] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 867a225140b94ed584a79744322953b7, State: Initialized, Role: FOLLOWER
I20260812 19:04:09.277143 2927 consensus_queue.cc:260] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d81caeda554d0796d3454b7a08ba83" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41779 } } peers { permanent_uuid: "867a225140b94ed584a79744322953b7" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40061 } } peers { permanent_uuid: "7aa537f1f40c4f53abe5e8e8adec26dc" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37323 } }
I20260812 19:04:09.278328 2927 ts_tablet_manager.cc:1428] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260812 19:04:09.278330 2929 raft_consensus.cc:348] T bc8f42d404ed4094ab116c0df6995b82 P b3d81caeda554d0796d3454b7a08ba83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d81caeda554d0796d3454b7a08ba83" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41779 } } peers { permanent_uuid: "867a225140b94ed584a79744322953b7" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40061 } } peers { permanent_uuid: "7aa537f1f40c4f53abe5e8e8adec26dc" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37323 } }
I20260812 19:04:09.278376 2928 raft_consensus.cc:348] T bc8f42d404ed4094ab116c0df6995b82 P 7aa537f1f40c4f53abe5e8e8adec26dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d81caeda554d0796d3454b7a08ba83" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41779 } } peers { permanent_uuid: "867a225140b94ed584a79744322953b7" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40061 } } peers { permanent_uuid: "7aa537f1f40c4f53abe5e8e8adec26dc" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37323 } }
I20260812 19:04:09.278765 2928 raft_consensus.cc:374] T bc8f42d404ed4094ab116c0df6995b82 P 7aa537f1f40c4f53abe5e8e8adec26dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:09.278820 2928 raft_consensus.cc:729] T bc8f42d404ed4094ab116c0df6995b82 P 7aa537f1f40c4f53abe5e8e8adec26dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aa537f1f40c4f53abe5e8e8adec26dc, State: Initialized, Role: FOLLOWER
I20260812 19:04:09.278820 2889 heartbeater.cc:503] Master 127.25.130.62:37435 was elected leader, sending a full tablet report...
I20260812 19:04:09.279012 2929 raft_consensus.cc:374] T bc8f42d404ed4094ab116c0df6995b82 P b3d81caeda554d0796d3454b7a08ba83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:09.279078 2929 raft_consensus.cc:729] T bc8f42d404ed4094ab116c0df6995b82 P b3d81caeda554d0796d3454b7a08ba83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3d81caeda554d0796d3454b7a08ba83, State: Initialized, Role: FOLLOWER
I20260812 19:04:09.279361 2928 consensus_queue.cc:260] T bc8f42d404ed4094ab116c0df6995b82 P 7aa537f1f40c4f53abe5e8e8adec26dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d81caeda554d0796d3454b7a08ba83" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41779 } } peers { permanent_uuid: "867a225140b94ed584a79744322953b7" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40061 } } peers { permanent_uuid: "7aa537f1f40c4f53abe5e8e8adec26dc" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37323 } }
I20260812 19:04:09.279532 2929 consensus_queue.cc:260] T bc8f42d404ed4094ab116c0df6995b82 P b3d81caeda554d0796d3454b7a08ba83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d81caeda554d0796d3454b7a08ba83" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41779 } } peers { permanent_uuid: "867a225140b94ed584a79744322953b7" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40061 } } peers { permanent_uuid: "7aa537f1f40c4f53abe5e8e8adec26dc" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37323 } }
I20260812 19:04:09.280270 2726 heartbeater.cc:503] Master 127.25.130.62:37435 was elected leader, sending a full tablet report...
I20260812 19:04:09.280493 2928 ts_tablet_manager.cc:1428] T bc8f42d404ed4094ab116c0df6995b82 P 7aa537f1f40c4f53abe5e8e8adec26dc: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260812 19:04:09.283434 2929 ts_tablet_manager.cc:1428] T bc8f42d404ed4094ab116c0df6995b82 P b3d81caeda554d0796d3454b7a08ba83: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260812 19:04:09.283943 2563 heartbeater.cc:503] Master 127.25.130.62:37435 was elected leader, sending a full tablet report...
I20260812 19:04:09.326299 2935 raft_consensus.cc:482] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:04:09.326505 2935 raft_consensus.cc:504] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d81caeda554d0796d3454b7a08ba83" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41779 } } peers { permanent_uuid: "867a225140b94ed584a79744322953b7" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40061 } } peers { permanent_uuid: "7aa537f1f40c4f53abe5e8e8adec26dc" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37323 } }
I20260812 19:04:09.327569 2935 leader_election.cc:288] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3d81caeda554d0796d3454b7a08ba83 (127.25.130.2:41779), 7aa537f1f40c4f53abe5e8e8adec26dc (127.25.130.1:37323)
I20260812 19:04:09.332242 2892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:09.327805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:37323 (local address 127.25.130.3:33151)
0812 19:04:09.328073 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:09.328085 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:09.328137 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:09.328356 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:09.328358 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:09.328370 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:09.328573 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:09.328578 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.329738 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.329741 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:09.330903 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:09.330915 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.331806 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.331810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:09.331906 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:09.331929 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 19:04:09.331991 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 19:04:09.332062 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":222}
I20260812 19:04:09.332250 2926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:09.327956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:33151 (local address 127.25.130.1:37323)
0812 19:04:09.328091 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:09.328095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:09.328116 (+ 21us) server_negotiation.cc:407] Connection header received
0812 19:04:09.328226 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:09.328229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:09.328275 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:09.328346 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:09.328897 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.329643 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.331031 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.331672 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.331805 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:09.331862 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:09.332032 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 19:04:09.332115 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 19:04:09.332378 2894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:09.327809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:41779 (local address 127.25.130.3:35367)
0812 19:04:09.327915 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:09.327930 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:09.328018 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:09.328318 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:09.328321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:09.328339 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:09.328569 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:09.328575 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.329577 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.329580 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:09.330903 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:09.330915 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.331933 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.331936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:09.332061 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:09.332086 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 19:04:09.332192 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 19:04:09.332268 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":48}
I20260812 19:04:09.332427 2923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:09.327956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:35367 (local address 127.25.130.2:41779)
0812 19:04:09.328085 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:09.328090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:09.328106 (+ 16us) server_negotiation.cc:407] Connection header received
0812 19:04:09.328164 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:09.328166 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:09.328219 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:09.328309 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:09.328685 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.329476 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.331156 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.331724 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.331825 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:09.331867 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:09.332224 (+ 357us) server_negotiation.cc:299] Negotiation successful
0812 19:04:09.332313 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 19:04:09.333289 2671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc8f42d404ed4094ab116c0df6995b82" candidate_uuid: "867a225140b94ed584a79744322953b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3d81caeda554d0796d3454b7a08ba83" is_pre_election: true
I20260812 19:04:09.333284 2511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc8f42d404ed4094ab116c0df6995b82" candidate_uuid: "867a225140b94ed584a79744322953b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aa537f1f40c4f53abe5e8e8adec26dc" is_pre_election: true
I20260812 19:04:09.333598 2671 raft_consensus.cc:2454] T bc8f42d404ed4094ab116c0df6995b82 P b3d81caeda554d0796d3454b7a08ba83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 867a225140b94ed584a79744322953b7 in term 0.
I20260812 19:04:09.333598 2511 raft_consensus.cc:2454] T bc8f42d404ed4094ab116c0df6995b82 P 7aa537f1f40c4f53abe5e8e8adec26dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 867a225140b94ed584a79744322953b7 in term 0.
I20260812 19:04:09.334093 2753 leader_election.cc:302] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7aa537f1f40c4f53abe5e8e8adec26dc, 867a225140b94ed584a79744322953b7, b3d81caeda554d0796d3454b7a08ba83; no voters:
I20260812 19:04:09.334355 2935 raft_consensus.cc:2781] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:04:09.334434 2935 raft_consensus.cc:482] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:04:09.334483 2935 raft_consensus.cc:3037] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:09.335568 2935 raft_consensus.cc:504] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d81caeda554d0796d3454b7a08ba83" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41779 } } peers { permanent_uuid: "867a225140b94ed584a79744322953b7" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40061 } } peers { permanent_uuid: "7aa537f1f40c4f53abe5e8e8adec26dc" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37323 } }
I20260812 19:04:09.335937 2935 leader_election.cc:288] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7 [CANDIDATE]: Term 1 election: Requested vote from peers b3d81caeda554d0796d3454b7a08ba83 (127.25.130.2:41779), 7aa537f1f40c4f53abe5e8e8adec26dc (127.25.130.1:37323)
I20260812 19:04:09.336310 2511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc8f42d404ed4094ab116c0df6995b82" candidate_uuid: "867a225140b94ed584a79744322953b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aa537f1f40c4f53abe5e8e8adec26dc"
I20260812 19:04:09.336427 2511 raft_consensus.cc:3037] T bc8f42d404ed4094ab116c0df6995b82 P 7aa537f1f40c4f53abe5e8e8adec26dc [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:09.337404 2511 raft_consensus.cc:2454] T bc8f42d404ed4094ab116c0df6995b82 P 7aa537f1f40c4f53abe5e8e8adec26dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 867a225140b94ed584a79744322953b7 in term 1.
I20260812 19:04:09.337684 2671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc8f42d404ed4094ab116c0df6995b82" candidate_uuid: "867a225140b94ed584a79744322953b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3d81caeda554d0796d3454b7a08ba83"
I20260812 19:04:09.337744 2752 leader_election.cc:302] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aa537f1f40c4f53abe5e8e8adec26dc, 867a225140b94ed584a79744322953b7; no voters:
I20260812 19:04:09.337828 2671 raft_consensus.cc:3037] T bc8f42d404ed4094ab116c0df6995b82 P b3d81caeda554d0796d3454b7a08ba83 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:09.337926 2935 raft_consensus.cc:2781] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:04:09.338140 2935 raft_consensus.cc:686] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7 [term 1 LEADER]: Becoming Leader. State: Replica: 867a225140b94ed584a79744322953b7, State: Running, Role: LEADER
I20260812 19:04:09.338506 2935 consensus_queue.cc:237] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d81caeda554d0796d3454b7a08ba83" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41779 } } peers { permanent_uuid: "867a225140b94ed584a79744322953b7" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40061 } } peers { permanent_uuid: "7aa537f1f40c4f53abe5e8e8adec26dc" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37323 } }
I20260812 19:04:09.339020 2671 raft_consensus.cc:2454] T bc8f42d404ed4094ab116c0df6995b82 P b3d81caeda554d0796d3454b7a08ba83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 867a225140b94ed584a79744322953b7 in term 1.
I20260812 19:04:09.341898 1376 catalog_manager.cc:5495] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 867a225140b94ed584a79744322953b7 (127.25.130.3). New cstate: current_term: 1 leader_uuid: "867a225140b94ed584a79744322953b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d81caeda554d0796d3454b7a08ba83" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "867a225140b94ed584a79744322953b7" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aa537f1f40c4f53abe5e8e8adec26dc" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37323 } health_report { overall_health: UNKNOWN } } }
I20260812 19:04:09.377624 2921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:09.374224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57774 (local address 127.25.130.3:40061)
0812 19:04:09.374361 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:09.374365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:09.374380 (+ 15us) server_negotiation.cc:407] Connection header received
0812 19:04:09.374476 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:09.374478 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:09.374536 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:09.374617 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:04:09.374818 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.375364 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.376212 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.376443 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.376539 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:09.376779 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:04:09.377240 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:04:09.377337 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:09.377367 (+ 30us) server_negotiation.cc:299] Negotiation successful
0812 19:04:09.377426 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 19:04:09.383985 2671 raft_consensus.cc:1261] T bc8f42d404ed4094ab116c0df6995b82 P b3d81caeda554d0796d3454b7a08ba83 [term 1 FOLLOWER]: Refusing update from remote peer 867a225140b94ed584a79744322953b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:04:09.383985 2511 raft_consensus.cc:1261] T bc8f42d404ed4094ab116c0df6995b82 P 7aa537f1f40c4f53abe5e8e8adec26dc [term 1 FOLLOWER]: Refusing update from remote peer 867a225140b94ed584a79744322953b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:04:09.384889 2935 consensus_queue.cc:1035] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aa537f1f40c4f53abe5e8e8adec26dc" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:04:09.385164 2939 consensus_queue.cc:1035] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3d81caeda554d0796d3454b7a08ba83" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:04:09.393967 2944 mvcc.cc:205] Tried to move back new op lower bound from 7317755696668717056 to 7317755696493543424. Current Snapshot: MvccSnapshot[applied={T|T < 7317755696668717056}]
I20260812 19:04:09.395098 2947 mvcc.cc:205] Tried to move back new op lower bound from 7317755696668717056 to 7317755696493543424. Current Snapshot: MvccSnapshot[applied={T|T < 7317755696668717056}]
I20260812 19:04:09.406033 1372 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:04:09.406363 1372 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:04:09.411000 1372 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dede9ac8627f4a27b237459ecc7d8ba3: Sending DeleteTablet for 3 replicas of tablet bc8f42d404ed4094ab116c0df6995b82
I20260812 19:04:09.412267 2488 tablet_service.cc:1514] Processing DeleteTablet for tablet bc8f42d404ed4094ab116c0df6995b82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:04:09 UTC) from {username='oryx', principal='oryx/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:46088
I20260812 19:04:09.412438 2814 tablet_service.cc:1514] Processing DeleteTablet for tablet bc8f42d404ed4094ab116c0df6995b82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:04:09 UTC) from {username='oryx', principal='oryx/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:57764
I20260812 19:04:09.413208 2648 tablet_service.cc:1514] Processing DeleteTablet for tablet bc8f42d404ed4094ab116c0df6995b82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:04:09 UTC) from {username='oryx', principal='oryx/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:37022
I20260812 19:04:09.413551 2956 tablet_replica.cc:330] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7: stopping tablet replica
I20260812 19:04:09.413951 2956 raft_consensus.cc:2229] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7 [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:04:09.414767 2956 raft_consensus.cc:2258] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:04:09.415345 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 1331
I20260812 19:04:09.420046 2956 ts_tablet_manager.cc:1905] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:04:09.423220 2956 ts_tablet_manager.cc:1918] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:04:09.423628 2959 tablet_replica.cc:330] T bc8f42d404ed4094ab116c0df6995b82 P b3d81caeda554d0796d3454b7a08ba83: stopping tablet replica
I20260812 19:04:09.424082 2959 raft_consensus.cc:2229] T bc8f42d404ed4094ab116c0df6995b82 P b3d81caeda554d0796d3454b7a08ba83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:04:09.424554 2959 raft_consensus.cc:2258] T bc8f42d404ed4094ab116c0df6995b82 P b3d81caeda554d0796d3454b7a08ba83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:04:09.424724 2956 log.cc:1196] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/wal/wals/bc8f42d404ed4094ab116c0df6995b82
I20260812 19:04:09.425391 2956 ts_tablet_manager.cc:1939] T bc8f42d404ed4094ab116c0df6995b82 P 867a225140b94ed584a79744322953b7: Deleting consensus metadata
I20260812 19:04:09.426729 2955 tablet_replica.cc:330] T bc8f42d404ed4094ab116c0df6995b82 P 7aa537f1f40c4f53abe5e8e8adec26dc: stopping tablet replica
I20260812 19:04:09.427120 2955 raft_consensus.cc:2229] T bc8f42d404ed4094ab116c0df6995b82 P 7aa537f1f40c4f53abe5e8e8adec26dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:04:09.427537 2955 raft_consensus.cc:2258] T bc8f42d404ed4094ab116c0df6995b82 P 7aa537f1f40c4f53abe5e8e8adec26dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:04:09.428243 2959 ts_tablet_manager.cc:1905] T bc8f42d404ed4094ab116c0df6995b82 P b3d81caeda554d0796d3454b7a08ba83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:04:09.428483 2902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:09.414156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54730 (local address 127.25.130.61:40369)
0812 19:04:09.414253 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:09.414258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:09.414381 (+ 123us) server_negotiation.cc:407] Connection header received
0812 19:04:09.414571 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:09.414574 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:09.414625 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:09.414671 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:09.419799 (+ 5128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.420761 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.421958 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.422325 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.422408 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:09.424534 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:09.424539 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:09.424542 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:09.424576 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:09.426390 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:09.426974 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:09.426978 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:09.426980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:09.427083 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:09.427431 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:09.427435 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:09.427437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:09.427681 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:09.427807 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:09.428163 (+ 356us) server_negotiation.cc:299] Negotiation successful
0812 19:04:09.428317 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 19:04:09.429282 2955 ts_tablet_manager.cc:1905] T bc8f42d404ed4094ab116c0df6995b82 P 7aa537f1f40c4f53abe5e8e8adec26dc: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 19:04:09.429577 2753 connection.cc:535] server connection from 127.0.0.1:57764 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 19:04:09.432435 2955 ts_tablet_manager.cc:1918] T bc8f42d404ed4094ab116c0df6995b82 P 7aa537f1f40c4f53abe5e8e8adec26dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:04:09.432565 2955 log.cc:1196] T bc8f42d404ed4094ab116c0df6995b82 P 7aa537f1f40c4f53abe5e8e8adec26dc: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-0/wal/wals/bc8f42d404ed4094ab116c0df6995b82
I20260812 19:04:09.432663 2959 ts_tablet_manager.cc:1918] T bc8f42d404ed4094ab116c0df6995b82 P b3d81caeda554d0796d3454b7a08ba83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:04:09.432857 2959 log.cc:1196] T bc8f42d404ed4094ab116c0df6995b82 P b3d81caeda554d0796d3454b7a08ba83: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/wal/wals/bc8f42d404ed4094ab116c0df6995b82
I20260812 19:04:09.432938 2955 ts_tablet_manager.cc:1939] T bc8f42d404ed4094ab116c0df6995b82 P 7aa537f1f40c4f53abe5e8e8adec26dc: Deleting consensus metadata
I20260812 19:04:09.433284 2959 ts_tablet_manager.cc:1939] T bc8f42d404ed4094ab116c0df6995b82 P b3d81caeda554d0796d3454b7a08ba83: Deleting consensus metadata
W20260812 19:04:09.434253 2435 connection.cc:441] server connection from 127.0.0.1:46088 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46088 (request call id 1) could send its response
W20260812 19:04:09.434321 2592 connection.cc:441] server connection from 127.0.0.1:37022 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37022 (request call id 1) could send its response
I20260812 19:04:09.439623 2570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:09.414210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34662 (local address 127.25.130.60:44715)
0812 19:04:09.414343 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:09.414348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:09.430406 (+ 16058us) server_negotiation.cc:407] Connection header received
0812 19:04:09.430616 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:09.430620 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:09.430680 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:09.430753 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:09.431807 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.432578 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.433718 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.434016 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.434101 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:09.435857 (+ 1756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:09.435861 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:09.435864 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:09.435899 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:09.437686 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:09.438220 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:09.438224 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:09.438226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:09.438324 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:09.438632 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:09.438635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:09.438637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:09.438852 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:09.438953 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:09.439315 (+ 362us) server_negotiation.cc:299] Negotiation successful
0812 19:04:09.439448 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 19:04:09.445380 2732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:09.434606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34666 (local address 127.25.130.60:44715)
0812 19:04:09.434727 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:09.434730 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:09.437533 (+ 2803us) server_negotiation.cc:407] Connection header received
0812 19:04:09.437584 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:09.437587 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:09.437641 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:09.437732 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:09.438460 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.439168 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.440305 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.440533 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.440618 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:09.442159 (+ 1541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:09.442163 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:09.442166 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:09.442198 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:09.443645 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:09.444048 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:09.444051 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:09.444053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:09.444136 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:09.444427 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:09.444430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:09.444432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:09.444646 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:09.444768 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:09.445085 (+ 317us) server_negotiation.cc:299] Negotiation successful
0812 19:04:09.445208 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 19:04:09.446664 2902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:09.433912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54736 (local address 127.25.130.61:40369)
0812 19:04:09.434027 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:09.434032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:09.435251 (+ 1219us) server_negotiation.cc:407] Connection header received
0812 19:04:09.435456 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:09.435460 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:09.435517 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:09.435609 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:09.436883 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.438187 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.439420 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:09.439774 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:09.439851 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:09.442517 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:09.442522 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:09.442524 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:09.442557 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:09.444066 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:09.444529 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:09.444532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:09.444534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:09.444629 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:09.444911 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:09.444914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:09.444916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:09.445127 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:09.445255 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:09.446360 (+ 1105us) server_negotiation.cc:299] Negotiation successful
0812 19:04:09.446500 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 19:04:10.397775 2967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:10.397153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.25.130.2:51369)
0812 19:04:10.397543 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:10.397652 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":101,"threads_started":1}
W20260812 19:04:10.398191 2726 heartbeater.cc:644] Failed to heartbeat to 127.25.130.62:37435 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
I20260812 19:04:10.398511 2969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:10.397955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.25.130.3:53295)
0812 19:04:10.398335 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:10.398428 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":68,"threads_started":1}
W20260812 19:04:10.398769 2889 heartbeater.cc:644] Failed to heartbeat to 127.25.130.62:37435 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
I20260812 19:04:10.402483 2971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:10.402099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.25.130.1:48363)
0812 19:04:10.402318 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:10.402398 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":65,"threads_started":1}
W20260812 19:04:10.402755 2563 heartbeater.cc:644] Failed to heartbeat to 127.25.130.62:37435 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
I20260812 19:04:10.911653 2972 raft_consensus.cc:482] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dede9ac8627f4a27b237459ecc7d8ba3)
I20260812 19:04:10.911876 2973 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dede9ac8627f4a27b237459ecc7d8ba3)
I20260812 19:04:10.911885 2972 raft_consensus.cc:504] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: false } }
I20260812 19:04:10.912052 2973 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: false } }
I20260812 19:04:10.912895 2972 leader_election.cc:288] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dede9ac8627f4a27b237459ecc7d8ba3 (127.25.130.62:37435), 5ef909f4f0644d49ad5990de41b6e21c (127.25.130.60:44715)
I20260812 19:04:10.913132 2973 leader_election.cc:288] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dede9ac8627f4a27b237459ecc7d8ba3 (127.25.130.62:37435), 501fbfc27f1746b39f928f6228d3aa5a (127.25.130.61:40369)
I20260812 19:04:10.913640 2974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:10.913072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.0.0.1:39150)
0812 19:04:10.913419 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:10.913530 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":73,"threads_started":1}
I20260812 19:04:10.913817 2975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:10.913071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.0.0.1:39152)
0812 19:04:10.913654 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:10.913728 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":73,"threads_started":1}
W20260812 19:04:10.913939 2124 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
W20260812 19:04:10.913983 2341 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
I20260812 19:04:10.914971 2974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:10.914722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.0.0.1:39178)
0812 19:04:10.914853 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:10.914902 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260812 19:04:10.914990 2975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:10.914736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.0.0.1:39168)
0812 19:04:10.914882 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:10.914927 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260812 19:04:10.915082 2124 leader_election.cc:334] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer dede9ac8627f4a27b237459ecc7d8ba3 (127.25.130.62:37435): Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
W20260812 19:04:10.915093 2341 leader_election.cc:334] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer dede9ac8627f4a27b237459ecc7d8ba3 (127.25.130.62:37435): Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
I20260812 19:04:10.917544 2978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:10.913428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:40369 (local address 127.0.0.1:54750)
0812 19:04:10.913662 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:10.913675 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:10.913774 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:10.914177 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:10.914181 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:10.914202 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:10.914317 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:10.914325 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:10.915537 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:10.915541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:10.916366 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:10.916375 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:10.917155 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:10.917158 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:10.917236 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:10.917258 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 19:04:10.917312 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 19:04:10.917388 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":48,"threads_started":1}
I20260812 19:04:10.917568 2979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:10.913552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54750 (local address 127.25.130.61:40369)
0812 19:04:10.913806 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:10.913810 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:10.913826 (+ 16us) server_negotiation.cc:407] Connection header received
0812 19:04:10.913877 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:10.913880 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:10.913925 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:10.914023 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:10.914484 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:10.915425 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:10.916482 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:10.917032 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:10.917131 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:10.917194 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:10.917347 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 19:04:10.917423 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":72,"threads_started":1}
I20260812 19:04:10.917815 2977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:10.913311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34670 (local address 127.25.130.60:44715)
0812 19:04:10.913543 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:10.913548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:10.914285 (+ 737us) server_negotiation.cc:407] Connection header received
0812 19:04:10.914505 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:10.914509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:10.914574 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:10.914632 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:10.915026 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:10.915658 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:10.916695 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:10.917268 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:10.917360 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:10.917413 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:10.917637 (+ 224us) server_negotiation.cc:299] Negotiation successful
0812 19:04:10.917709 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":69,"threads_started":1}
I20260812 19:04:10.917825 2976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:10.913071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:44715 (local address 127.0.0.1:34670)
0812 19:04:10.914208 (+ 1137us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:10.914221 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:10.914381 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:10.914791 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:10.914794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:10.914812 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:10.914902 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:10.914908 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:10.915786 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:10.915788 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:10.916587 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:10.916597 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:10.917383 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:10.917387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:10.917504 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:10.917528 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 19:04:10.917607 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 19:04:10.917685 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1075,"mutex_wait_us":209,"thread_start_us":49,"threads_started":1}
I20260812 19:04:10.918275 2157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5ef909f4f0644d49ad5990de41b6e21c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "501fbfc27f1746b39f928f6228d3aa5a" is_pre_election: true
I20260812 19:04:10.918299 2365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "501fbfc27f1746b39f928f6228d3aa5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5ef909f4f0644d49ad5990de41b6e21c" is_pre_election: true
I20260812 19:04:10.918478 2365 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 501fbfc27f1746b39f928f6228d3aa5a in term 1.
I20260812 19:04:10.918478 2157 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ef909f4f0644d49ad5990de41b6e21c in term 1.
I20260812 19:04:10.918828 2123 leader_election.cc:302] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 501fbfc27f1746b39f928f6228d3aa5a, 5ef909f4f0644d49ad5990de41b6e21c; no voters: dede9ac8627f4a27b237459ecc7d8ba3
I20260812 19:04:10.918826 2340 leader_election.cc:302] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 501fbfc27f1746b39f928f6228d3aa5a, 5ef909f4f0644d49ad5990de41b6e21c; no voters: dede9ac8627f4a27b237459ecc7d8ba3
I20260812 19:04:10.919032 2972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 19:04:10.919054 2973 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 19:04:10.919088 2972 raft_consensus.cc:482] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 1 FOLLOWER]: Starting leader election (detected failure of leader dede9ac8627f4a27b237459ecc7d8ba3)
I20260812 19:04:10.919103 2973 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [term 1 FOLLOWER]: Starting leader election (detected failure of leader dede9ac8627f4a27b237459ecc7d8ba3)
I20260812 19:04:10.919128 2972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:04:10.919140 2973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:04:10.920150 2972 raft_consensus.cc:504] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: false } }
I20260812 19:04:10.920148 2973 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: false } }
I20260812 19:04:10.920475 2972 leader_election.cc:288] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [CANDIDATE]: Term 2 election: Requested vote from peers dede9ac8627f4a27b237459ecc7d8ba3 (127.25.130.62:37435), 5ef909f4f0644d49ad5990de41b6e21c (127.25.130.60:44715)
I20260812 19:04:10.920488 2973 leader_election.cc:288] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [CANDIDATE]: Term 2 election: Requested vote from peers dede9ac8627f4a27b237459ecc7d8ba3 (127.25.130.62:37435), 501fbfc27f1746b39f928f6228d3aa5a (127.25.130.61:40369)
I20260812 19:04:10.920929 2976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:10.920639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.0.0.1:39182)
0812 19:04:10.920807 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:10.920860 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260812 19:04:10.920917 2157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5ef909f4f0644d49ad5990de41b6e21c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "501fbfc27f1746b39f928f6228d3aa5a"
I20260812 19:04:10.920970 2365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "501fbfc27f1746b39f928f6228d3aa5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5ef909f4f0644d49ad5990de41b6e21c"
I20260812 19:04:10.921113 2365 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 501fbfc27f1746b39f928f6228d3aa5a in current term 2: Already voted for candidate 5ef909f4f0644d49ad5990de41b6e21c in this term.
I20260812 19:04:10.921113 2157 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5ef909f4f0644d49ad5990de41b6e21c in current term 2: Already voted for candidate 501fbfc27f1746b39f928f6228d3aa5a in this term.
I20260812 19:04:10.921383 2978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:10.920695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.0.0.1:39186)
0812 19:04:10.920821 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:10.920875 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 19:04:10.921667 2976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:10.921421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.0.0.1:39194)
0812 19:04:10.921557 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:10.921607 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260812 19:04:10.921919 2124 leader_election.cc:334] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer dede9ac8627f4a27b237459ecc7d8ba3 (127.25.130.62:37435): Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
I20260812 19:04:10.921990 2978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:10.921807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.0.0.1:39208)
0812 19:04:10.921900 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:10.921946 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 19:04:10.921985 2124 leader_election.cc:302] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 501fbfc27f1746b39f928f6228d3aa5a; no voters: 5ef909f4f0644d49ad5990de41b6e21c, dede9ac8627f4a27b237459ecc7d8ba3
W20260812 19:04:10.922065 2341 leader_election.cc:334] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer dede9ac8627f4a27b237459ecc7d8ba3 (127.25.130.62:37435): Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
I20260812 19:04:10.922111 2972 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 19:04:10.922119 2341 leader_election.cc:302] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ef909f4f0644d49ad5990de41b6e21c; no voters: 501fbfc27f1746b39f928f6228d3aa5a, dede9ac8627f4a27b237459ecc7d8ba3
I20260812 19:04:10.922255 2973 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 19:04:11.399621 2982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:11.398893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.25.130.2:47929)
0812 19:04:11.399421 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:11.399502 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":127,"threads_started":1}
I20260812 19:04:11.399964 2983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:11.399369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.25.130.3:49547)
0812 19:04:11.399689 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:11.399850 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":94,"threads_started":1}
I20260812 19:04:11.403668 2984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:11.403261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.25.130.1:45571)
0812 19:04:11.403513 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:11.403580 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":93,"threads_started":1}
I20260812 19:04:12.401648 2986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.400761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.25.130.3:49759)
0812 19:04:12.401397 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:12.401500 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":123,"threads_started":1}
I20260812 19:04:12.401647 2985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.400761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.25.130.2:58093)
0812 19:04:12.401387 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:12.401501 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":114,"threads_started":1}
I20260812 19:04:12.404829 2987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.404378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.25.130.1:59607)
0812 19:04:12.404640 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:12.404712 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":84,"threads_started":1}
I20260812 19:04:12.556572 2988 raft_consensus.cc:482] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:04:12.556787 2988 raft_consensus.cc:504] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: false } }
I20260812 19:04:12.557372 2988 leader_election.cc:288] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dede9ac8627f4a27b237459ecc7d8ba3 (127.25.130.62:37435), 5ef909f4f0644d49ad5990de41b6e21c (127.25.130.60:44715)
I20260812 19:04:12.557996 2365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "501fbfc27f1746b39f928f6228d3aa5a" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5ef909f4f0644d49ad5990de41b6e21c" is_pre_election: true
I20260812 19:04:12.558184 2365 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 501fbfc27f1746b39f928f6228d3aa5a in term 2.
I20260812 19:04:12.558193 2989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.557676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.0.0.1:39472)
0812 19:04:12.557975 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:12.558056 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":105,"threads_started":1}
I20260812 19:04:12.558568 2123 leader_election.cc:302] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 501fbfc27f1746b39f928f6228d3aa5a, 5ef909f4f0644d49ad5990de41b6e21c; no voters:
I20260812 19:04:12.558779 2988 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:04:12.558852 2988 raft_consensus.cc:482] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:04:12.558893 2988 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:04:12.559293 2989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.559029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.0.0.1:39488)
0812 19:04:12.559160 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:12.559222 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 19:04:12.559403 2124 leader_election.cc:334] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer dede9ac8627f4a27b237459ecc7d8ba3 (127.25.130.62:37435): Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
I20260812 19:04:12.560338 2988 raft_consensus.cc:504] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: false } }
I20260812 19:04:12.560676 2988 leader_election.cc:288] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [CANDIDATE]: Term 3 election: Requested vote from peers dede9ac8627f4a27b237459ecc7d8ba3 (127.25.130.62:37435), 5ef909f4f0644d49ad5990de41b6e21c (127.25.130.60:44715)
I20260812 19:04:12.561014 2989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.560801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.0.0.1:39498)
0812 19:04:12.560901 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:12.560951 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 19:04:12.561074 2365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "501fbfc27f1746b39f928f6228d3aa5a" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5ef909f4f0644d49ad5990de41b6e21c"
I20260812 19:04:12.561241 2365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:04:12.561676 2989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.561490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.0.0.1:39502)
0812 19:04:12.561575 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:12.561624 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260812 19:04:12.561784 2124 leader_election.cc:334] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer dede9ac8627f4a27b237459ecc7d8ba3 (127.25.130.62:37435): Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
I20260812 19:04:12.562577 2365 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 501fbfc27f1746b39f928f6228d3aa5a in term 3.
I20260812 19:04:12.562887 2123 leader_election.cc:302] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 501fbfc27f1746b39f928f6228d3aa5a, 5ef909f4f0644d49ad5990de41b6e21c; no voters: dede9ac8627f4a27b237459ecc7d8ba3
I20260812 19:04:12.563066 2988 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:04:12.563244 2988 raft_consensus.cc:686] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [term 3 LEADER]: Becoming Leader. State: Replica: 501fbfc27f1746b39f928f6228d3aa5a, State: Running, Role: LEADER
I20260812 19:04:12.563567 2988 consensus_queue.cc:237] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: false } }
I20260812 19:04:12.564728 2991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 501fbfc27f1746b39f928f6228d3aa5a. Latest consensus state: current_term: 3 leader_uuid: "501fbfc27f1746b39f928f6228d3aa5a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: false } } }
I20260812 19:04:12.564862 2991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:12.565196 2993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:04:12.567894 2993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:04:12.568288 2993 catalog_manager.cc:1259] Loaded cluster ID: a674a8bb57f4493194ed80aa887c9dca
I20260812 19:04:12.568337 2993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:04:12.568965 2993 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:04:12.569350 2993 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a: Loaded TSK: 0
I20260812 19:04:12.570186 2993 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:04:12.588011 2139 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54736:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:04:12.591794 2139 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:04:12.595372 2989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.595100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.0.0.1:39520)
0812 19:04:12.595224 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:12.595285 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 19:04:12.595551 2365 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [term 3 FOLLOWER]: Refusing update from remote peer 501fbfc27f1746b39f928f6228d3aa5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260812 19:04:12.596186 2989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.595894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.0.0.1:39534)
0812 19:04:12.596048 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:12.596108 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 19:04:12.596143 2991 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260812 19:04:12.596321 2124 consensus_peers.cc:487] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a -> Peer dede9ac8627f4a27b237459ecc7d8ba3 (127.25.130.62:37435): Couldn't send request to peer dede9ac8627f4a27b237459ecc7d8ba3. Status: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 19:04:12.597942 2995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 501fbfc27f1746b39f928f6228d3aa5a. Latest consensus state: current_term: 3 leader_uuid: "501fbfc27f1746b39f928f6228d3aa5a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: false } } }
I20260812 19:04:12.598106 2995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:04:12.599457 2991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "501fbfc27f1746b39f928f6228d3aa5a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: false } } }
I20260812 19:04:12.599596 2991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:12.599756 2995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "501fbfc27f1746b39f928f6228d3aa5a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dede9ac8627f4a27b237459ecc7d8ba3" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37435 } } peers { permanent_uuid: "501fbfc27f1746b39f928f6228d3aa5a" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 40369 } attrs { promote: false } } peers { permanent_uuid: "5ef909f4f0644d49ad5990de41b6e21c" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 44715 } attrs { promote: false } } }
I20260812 19:04:12.599897 2995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ef909f4f0644d49ad5990de41b6e21c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:04:12.611626 3002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.606901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:37323 (local address 127.0.0.1:46100)
0812 19:04:12.607157 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:12.607168 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:12.607255 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:12.608350 (+ 1095us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:12.608353 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:12.608368 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:12.608465 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:12.608471 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:12.609219 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:12.609223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:12.610356 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:12.610367 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:12.611150 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:12.611155 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:12.611247 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:12.611276 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 19:04:12.611350 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 19:04:12.611435 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":47,"thread_start_us":119,"threads_started":1}
I20260812 19:04:12.611660 3003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.607100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46100 (local address 127.25.130.1:37323)
0812 19:04:12.608107 (+ 1007us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:12.608112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:12.608129 (+ 17us) server_negotiation.cc:407] Connection header received
0812 19:04:12.608190 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:12.608194 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:12.608251 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:12.608339 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:12.608576 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:12.609108 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:12.610482 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:12.611034 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:12.611139 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:12.611219 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:12.611385 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 19:04:12.611486 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":911,"thread_start_us":98,"threads_started":1}
I20260812 19:04:12.611999 2989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.606335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:41779 (local address 127.0.0.1:37034)
0812 19:04:12.606477 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:12.606494 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:12.606597 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:12.607875 (+ 1278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:12.607878 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:12.607896 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:12.608034 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:12.608041 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:12.609582 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:12.609585 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:12.610697 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:12.610704 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:12.611600 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:12.611604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:12.611686 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:12.611738 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 19:04:12.611790 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 19:04:12.611908 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":3}
I20260812 19:04:12.612391 2488 tablet_service.cc:1467] Processing CreateTablet for tablet 8770d8ab6ba3446697c0ac8058f0ca2b (DEFAULT_TABLE table=test-table [id=0ef585d4d3fe48968a48049272d3b628]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:04:12.612531 3001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.606534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37034 (local address 127.25.130.2:41779)
0812 19:04:12.607635 (+ 1101us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:12.607641 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:12.607657 (+ 16us) server_negotiation.cc:407] Connection header received
0812 19:04:12.607720 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:12.607722 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:12.607770 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:12.607847 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:12.608175 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:12.609423 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:12.610839 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:12.611414 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:12.612174 (+ 760us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:12.612245 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:12.612280 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 19:04:12.612394 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":998,"thread_start_us":133,"threads_started":1}
I20260812 19:04:12.612798 2488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8770d8ab6ba3446697c0ac8058f0ca2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:12.612888 3005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.607470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57776 (local address 127.25.130.3:40061)
0812 19:04:12.607757 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:12.607761 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:12.607773 (+ 12us) server_negotiation.cc:407] Connection header received
0812 19:04:12.607842 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:12.607845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:12.607901 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:12.607997 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:12.608274 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:12.609114 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:12.610779 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:12.611431 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:12.611542 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:12.611620 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:12.612642 (+ 1022us) server_negotiation.cc:299] Negotiation successful
0812 19:04:12.612739 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":95,"threads_started":1}
I20260812 19:04:12.613152 3004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.607275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:40061 (local address 127.0.0.1:57776)
0812 19:04:12.607494 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:12.607503 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:12.607576 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:12.608034 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:12.608036 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:12.608055 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:12.608156 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:12.608163 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:12.609253 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:12.609257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:12.610641 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:12.610653 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:12.611682 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:12.611685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:12.612308 (+ 623us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:12.612331 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 19:04:12.612404 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 19:04:12.612480 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":595,"thread_start_us":99,"threads_started":1}
I20260812 19:04:12.613919 2814 tablet_service.cc:1467] Processing CreateTablet for tablet 8770d8ab6ba3446697c0ac8058f0ca2b (DEFAULT_TABLE table=test-table [id=0ef585d4d3fe48968a48049272d3b628]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:04:12.614233 2814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8770d8ab6ba3446697c0ac8058f0ca2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:12.616631 3006 tablet_bootstrap.cc:492] T 8770d8ab6ba3446697c0ac8058f0ca2b P 7aa537f1f40c4f53abe5e8e8adec26dc: Bootstrap starting.
I20260812 19:04:12.616602 2648 tablet_service.cc:1467] Processing CreateTablet for tablet 8770d8ab6ba3446697c0ac8058f0ca2b (DEFAULT_TABLE table=test-table [id=0ef585d4d3fe48968a48049272d3b628]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:04:12.616915 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8770d8ab6ba3446697c0ac8058f0ca2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:12.617951 3006 tablet_bootstrap.cc:654] T 8770d8ab6ba3446697c0ac8058f0ca2b P 7aa537f1f40c4f53abe5e8e8adec26dc: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:12.618613 3008 tablet_bootstrap.cc:492] T 8770d8ab6ba3446697c0ac8058f0ca2b P 867a225140b94ed584a79744322953b7: Bootstrap starting.
I20260812 19:04:12.619246 3006 tablet_bootstrap.cc:492] T 8770d8ab6ba3446697c0ac8058f0ca2b P 7aa537f1f40c4f53abe5e8e8adec26dc: No bootstrap required, opened a new log
I20260812 19:04:12.619411 3012 tablet_bootstrap.cc:492] T 8770d8ab6ba3446697c0ac8058f0ca2b P b3d81caeda554d0796d3454b7a08ba83: Bootstrap starting.
I20260812 19:04:12.619513 3008 tablet_bootstrap.cc:654] T 8770d8ab6ba3446697c0ac8058f0ca2b P 867a225140b94ed584a79744322953b7: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:12.619459 3006 ts_tablet_manager.cc:1397] T 8770d8ab6ba3446697c0ac8058f0ca2b P 7aa537f1f40c4f53abe5e8e8adec26dc: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 19:04:12.620061 3006 raft_consensus.cc:348] T 8770d8ab6ba3446697c0ac8058f0ca2b P 7aa537f1f40c4f53abe5e8e8adec26dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d81caeda554d0796d3454b7a08ba83" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41779 } } peers { permanent_uuid: "867a225140b94ed584a79744322953b7" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40061 } } peers { permanent_uuid: "7aa537f1f40c4f53abe5e8e8adec26dc" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37323 } }
I20260812 19:04:12.620195 3006 raft_consensus.cc:374] T 8770d8ab6ba3446697c0ac8058f0ca2b P 7aa537f1f40c4f53abe5e8e8adec26dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:12.620234 3006 raft_consensus.cc:729] T 8770d8ab6ba3446697c0ac8058f0ca2b P 7aa537f1f40c4f53abe5e8e8adec26dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aa537f1f40c4f53abe5e8e8adec26dc, State: Initialized, Role: FOLLOWER
I20260812 19:04:12.620352 3012 tablet_bootstrap.cc:654] T 8770d8ab6ba3446697c0ac8058f0ca2b P b3d81caeda554d0796d3454b7a08ba83: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:12.620373 3006 consensus_queue.cc:260] T 8770d8ab6ba3446697c0ac8058f0ca2b P 7aa537f1f40c4f53abe5e8e8adec26dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d81caeda554d0796d3454b7a08ba83" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41779 } } peers { permanent_uuid: "867a225140b94ed584a79744322953b7" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40061 } } peers { permanent_uuid: "7aa537f1f40c4f53abe5e8e8adec26dc" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37323 } }
I20260812 19:04:12.620779 3006 ts_tablet_manager.cc:1428] T 8770d8ab6ba3446697c0ac8058f0ca2b P 7aa537f1f40c4f53abe5e8e8adec26dc: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 19:04:12.621126 3008 tablet_bootstrap.cc:492] T 8770d8ab6ba3446697c0ac8058f0ca2b P 867a225140b94ed584a79744322953b7: No bootstrap required, opened a new log
I20260812 19:04:12.621214 3008 ts_tablet_manager.cc:1397] T 8770d8ab6ba3446697c0ac8058f0ca2b P 867a225140b94ed584a79744322953b7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 19:04:12.621649 3008 raft_consensus.cc:348] T 8770d8ab6ba3446697c0ac8058f0ca2b P 867a225140b94ed584a79744322953b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d81caeda554d0796d3454b7a08ba83" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41779 } } peers { permanent_uuid: "867a225140b94ed584a79744322953b7" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40061 } } peers { permanent_uuid: "7aa537f1f40c4f53abe5e8e8adec26dc" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37323 } }
I20260812 19:04:12.621769 3008 raft_consensus.cc:374] T 8770d8ab6ba3446697c0ac8058f0ca2b P 867a225140b94ed584a79744322953b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:12.621807 3008 raft_consensus.cc:729] T 8770d8ab6ba3446697c0ac8058f0ca2b P 867a225140b94ed584a79744322953b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 867a225140b94ed584a79744322953b7, State: Initialized, Role: FOLLOWER
I20260812 19:04:12.621933 3008 consensus_queue.cc:260] T 8770d8ab6ba3446697c0ac8058f0ca2b P 867a225140b94ed584a79744322953b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d81caeda554d0796d3454b7a08ba83" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41779 } } peers { permanent_uuid: "867a225140b94ed584a79744322953b7" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40061 } } peers { permanent_uuid: "7aa537f1f40c4f53abe5e8e8adec26dc" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37323 } }
I20260812 19:04:12.622299 3008 ts_tablet_manager.cc:1428] T 8770d8ab6ba3446697c0ac8058f0ca2b P 867a225140b94ed584a79744322953b7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 19:04:12.623145 2986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.622876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.25.130.3:44603)
0812 19:04:12.622998 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:12.623066 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 19:04:12.623493 2987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.622776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.25.130.1:49339)
0812 19:04:12.623354 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:12.623417 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487}
I20260812 19:04:12.623615 3012 tablet_bootstrap.cc:492] T 8770d8ab6ba3446697c0ac8058f0ca2b P b3d81caeda554d0796d3454b7a08ba83: No bootstrap required, opened a new log
I20260812 19:04:12.623716 3012 ts_tablet_manager.cc:1397] T 8770d8ab6ba3446697c0ac8058f0ca2b P b3d81caeda554d0796d3454b7a08ba83: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 19:04:12.624169 3012 raft_consensus.cc:348] T 8770d8ab6ba3446697c0ac8058f0ca2b P b3d81caeda554d0796d3454b7a08ba83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d81caeda554d0796d3454b7a08ba83" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41779 } } peers { permanent_uuid: "867a225140b94ed584a79744322953b7" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40061 } } peers { permanent_uuid: "7aa537f1f40c4f53abe5e8e8adec26dc" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37323 } }
I20260812 19:04:12.624289 3012 raft_consensus.cc:374] T 8770d8ab6ba3446697c0ac8058f0ca2b P b3d81caeda554d0796d3454b7a08ba83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:12.624321 3012 raft_consensus.cc:729] T 8770d8ab6ba3446697c0ac8058f0ca2b P b3d81caeda554d0796d3454b7a08ba83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3d81caeda554d0796d3454b7a08ba83, State: Initialized, Role: FOLLOWER
I20260812 19:04:12.624472 3012 consensus_queue.cc:260] T 8770d8ab6ba3446697c0ac8058f0ca2b P b3d81caeda554d0796d3454b7a08ba83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d81caeda554d0796d3454b7a08ba83" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41779 } } peers { permanent_uuid: "867a225140b94ed584a79744322953b7" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40061 } } peers { permanent_uuid: "7aa537f1f40c4f53abe5e8e8adec26dc" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37323 } }
I20260812 19:04:12.624867 3012 ts_tablet_manager.cc:1428] T 8770d8ab6ba3446697c0ac8058f0ca2b P b3d81caeda554d0796d3454b7a08ba83: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 19:04:12.625102 2986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.624882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.25.130.3:58059)
0812 19:04:12.624981 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:12.625033 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 19:04:12.625553 2564 heartbeater.cc:503] Master 127.25.130.61:40369 was elected leader, sending a full tablet report...
I20260812 19:04:12.625938 2987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.625723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.25.130.1:39419)
0812 19:04:12.625827 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:12.625877 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 19:04:12.626469 2985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.626179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.25.130.2:38893)
0812 19:04:12.626342 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:12.626395 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260812 19:04:12.627455 2985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.627256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.25.130.2:43821)
0812 19:04:12.627354 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:12.627399 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 19:04:12.628074 2727 heartbeater.cc:503] Master 127.25.130.61:40369 was elected leader, sending a full tablet report...
W20260812 19:04:12.701246 2891 tablet.cc:2369] T 8770d8ab6ba3446697c0ac8058f0ca2b P 867a225140b94ed584a79744322953b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 19:04:12.732300 2728 tablet.cc:2369] T 8770d8ab6ba3446697c0ac8058f0ca2b P b3d81caeda554d0796d3454b7a08ba83: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 19:04:12.781798 2565 tablet.cc:2369] T 8770d8ab6ba3446697c0ac8058f0ca2b P 7aa537f1f40c4f53abe5e8e8adec26dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:04:12.926352 3014 raft_consensus.cc:482] T 8770d8ab6ba3446697c0ac8058f0ca2b P 7aa537f1f40c4f53abe5e8e8adec26dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:04:12.926532 3014 raft_consensus.cc:504] T 8770d8ab6ba3446697c0ac8058f0ca2b P 7aa537f1f40c4f53abe5e8e8adec26dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d81caeda554d0796d3454b7a08ba83" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41779 } } peers { permanent_uuid: "867a225140b94ed584a79744322953b7" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40061 } } peers { permanent_uuid: "7aa537f1f40c4f53abe5e8e8adec26dc" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37323 } }
I20260812 19:04:12.927790 3014 leader_election.cc:288] T 8770d8ab6ba3446697c0ac8058f0ca2b P 7aa537f1f40c4f53abe5e8e8adec26dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3d81caeda554d0796d3454b7a08ba83 (127.25.130.2:41779), 867a225140b94ed584a79744322953b7 (127.25.130.3:40061)
I20260812 19:04:12.932175 2987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.928090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:41779 (local address 127.25.130.1:51763)
0812 19:04:12.928225 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:12.928241 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:12.928362 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:12.928653 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:12.928656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:12.928681 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:12.928835 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:12.928842 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:12.929665 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:12.929668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:12.930755 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:12.930766 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:12.931616 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:12.931619 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:12.931774 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:12.931806 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 19:04:12.931883 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 19:04:12.931960 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 19:04:12.932228 3001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.928190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:51763 (local address 127.25.130.2:41779)
0812 19:04:12.928387 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:12.928392 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:12.928410 (+ 18us) server_negotiation.cc:407] Connection header received
0812 19:04:12.928480 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:12.928483 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:12.928537 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:12.928621 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:12.929002 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:12.929565 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:12.930906 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:12.931507 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:12.931604 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:12.931655 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:12.931967 (+ 312us) server_negotiation.cc:299] Negotiation successful
0812 19:04:12.932078 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260812 19:04:12.932663 3019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.928090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:40061 (local address 127.25.130.1:59915)
0812 19:04:12.928847 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:12.928860 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:12.928951 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:12.929221 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:12.929224 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:12.929242 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:12.929350 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:12.929356 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:12.930323 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:12.930326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:12.931271 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:12.931283 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:12.932165 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:12.932169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:12.932276 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:12.932300 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 19:04:12.932382 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 19:04:12.932457 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"mutex_wait_us":342,"thread_start_us":187,"threads_started":1}
I20260812 19:04:12.932683 3005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.928194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:59915 (local address 127.25.130.3:40061)
0812 19:04:12.928363 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:12.928367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:12.928937 (+ 570us) server_negotiation.cc:407] Connection header received
0812 19:04:12.929079 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:12.929083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:12.929133 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:12.929198 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:12.929479 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:12.930201 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:12.931378 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:12.931953 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:12.932061 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:12.932130 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:12.932462 (+ 332us) server_negotiation.cc:299] Negotiation successful
0812 19:04:12.932551 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 19:04:12.932945 2671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8770d8ab6ba3446697c0ac8058f0ca2b" candidate_uuid: "7aa537f1f40c4f53abe5e8e8adec26dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3d81caeda554d0796d3454b7a08ba83" is_pre_election: true
I20260812 19:04:12.933130 2671 raft_consensus.cc:2454] T 8770d8ab6ba3446697c0ac8058f0ca2b P b3d81caeda554d0796d3454b7a08ba83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aa537f1f40c4f53abe5e8e8adec26dc in term 0.
I20260812 19:04:12.933382 2837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8770d8ab6ba3446697c0ac8058f0ca2b" candidate_uuid: "7aa537f1f40c4f53abe5e8e8adec26dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "867a225140b94ed584a79744322953b7" is_pre_election: true
I20260812 19:04:12.933585 2433 leader_election.cc:302] T 8770d8ab6ba3446697c0ac8058f0ca2b P 7aa537f1f40c4f53abe5e8e8adec26dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aa537f1f40c4f53abe5e8e8adec26dc, b3d81caeda554d0796d3454b7a08ba83; no voters:
I20260812 19:04:12.933653 2837 raft_consensus.cc:2454] T 8770d8ab6ba3446697c0ac8058f0ca2b P 867a225140b94ed584a79744322953b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aa537f1f40c4f53abe5e8e8adec26dc in term 0.
I20260812 19:04:12.933851 3014 raft_consensus.cc:2781] T 8770d8ab6ba3446697c0ac8058f0ca2b P 7aa537f1f40c4f53abe5e8e8adec26dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:04:12.933913 3014 raft_consensus.cc:482] T 8770d8ab6ba3446697c0ac8058f0ca2b P 7aa537f1f40c4f53abe5e8e8adec26dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:04:12.933951 3014 raft_consensus.cc:3037] T 8770d8ab6ba3446697c0ac8058f0ca2b P 7aa537f1f40c4f53abe5e8e8adec26dc [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:12.935303 3014 raft_consensus.cc:504] T 8770d8ab6ba3446697c0ac8058f0ca2b P 7aa537f1f40c4f53abe5e8e8adec26dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d81caeda554d0796d3454b7a08ba83" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41779 } } peers { permanent_uuid: "867a225140b94ed584a79744322953b7" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40061 } } peers { permanent_uuid: "7aa537f1f40c4f53abe5e8e8adec26dc" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37323 } }
I20260812 19:04:12.935638 3014 leader_election.cc:288] T 8770d8ab6ba3446697c0ac8058f0ca2b P 7aa537f1f40c4f53abe5e8e8adec26dc [CANDIDATE]: Term 1 election: Requested vote from peers b3d81caeda554d0796d3454b7a08ba83 (127.25.130.2:41779), 867a225140b94ed584a79744322953b7 (127.25.130.3:40061)
I20260812 19:04:12.935988 2671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8770d8ab6ba3446697c0ac8058f0ca2b" candidate_uuid: "7aa537f1f40c4f53abe5e8e8adec26dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3d81caeda554d0796d3454b7a08ba83"
I20260812 19:04:12.936015 2837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8770d8ab6ba3446697c0ac8058f0ca2b" candidate_uuid: "7aa537f1f40c4f53abe5e8e8adec26dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "867a225140b94ed584a79744322953b7"
I20260812 19:04:12.936103 2671 raft_consensus.cc:3037] T 8770d8ab6ba3446697c0ac8058f0ca2b P b3d81caeda554d0796d3454b7a08ba83 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:12.936133 2837 raft_consensus.cc:3037] T 8770d8ab6ba3446697c0ac8058f0ca2b P 867a225140b94ed584a79744322953b7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:12.937126 2671 raft_consensus.cc:2454] T 8770d8ab6ba3446697c0ac8058f0ca2b P b3d81caeda554d0796d3454b7a08ba83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aa537f1f40c4f53abe5e8e8adec26dc in term 1.
I20260812 19:04:12.937193 2837 raft_consensus.cc:2454] T 8770d8ab6ba3446697c0ac8058f0ca2b P 867a225140b94ed584a79744322953b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aa537f1f40c4f53abe5e8e8adec26dc in term 1.
I20260812 19:04:12.937422 2433 leader_election.cc:302] T 8770d8ab6ba3446697c0ac8058f0ca2b P 7aa537f1f40c4f53abe5e8e8adec26dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aa537f1f40c4f53abe5e8e8adec26dc, b3d81caeda554d0796d3454b7a08ba83; no voters:
I20260812 19:04:12.937626 3014 raft_consensus.cc:2781] T 8770d8ab6ba3446697c0ac8058f0ca2b P 7aa537f1f40c4f53abe5e8e8adec26dc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:04:12.937842 3014 raft_consensus.cc:686] T 8770d8ab6ba3446697c0ac8058f0ca2b P 7aa537f1f40c4f53abe5e8e8adec26dc [term 1 LEADER]: Becoming Leader. State: Replica: 7aa537f1f40c4f53abe5e8e8adec26dc, State: Running, Role: LEADER
I20260812 19:04:12.938130 3014 consensus_queue.cc:237] T 8770d8ab6ba3446697c0ac8058f0ca2b P 7aa537f1f40c4f53abe5e8e8adec26dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d81caeda554d0796d3454b7a08ba83" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41779 } } peers { permanent_uuid: "867a225140b94ed584a79744322953b7" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40061 } } peers { permanent_uuid: "7aa537f1f40c4f53abe5e8e8adec26dc" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37323 } }
I20260812 19:04:12.938581 3019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.938330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.25.130.1:48233)
0812 19:04:12.938454 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:12.938507 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 19:04:12.940778 2139 catalog_manager.cc:5495] T 8770d8ab6ba3446697c0ac8058f0ca2b P 7aa537f1f40c4f53abe5e8e8adec26dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aa537f1f40c4f53abe5e8e8adec26dc (127.25.130.1). New cstate: current_term: 1 leader_uuid: "7aa537f1f40c4f53abe5e8e8adec26dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d81caeda554d0796d3454b7a08ba83" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "867a225140b94ed584a79744322953b7" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aa537f1f40c4f53abe5e8e8adec26dc" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37323 } health_report { overall_health: HEALTHY } } }
I20260812 19:04:12.979439 3003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.975795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46110 (local address 127.25.130.1:37323)
0812 19:04:12.975918 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:12.975922 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:12.975936 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:04:12.976037 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:12.976040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:12.976109 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:12.976221 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:04:12.976509 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:12.977018 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:12.977912 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:12.978147 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:12.978237 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:12.978500 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:04:12.979034 (+ 534us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:04:12.979156 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:12.979229 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 19:04:12.979294 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 19:04:12.982681 2671 raft_consensus.cc:1261] T 8770d8ab6ba3446697c0ac8058f0ca2b P b3d81caeda554d0796d3454b7a08ba83 [term 1 FOLLOWER]: Refusing update from remote peer 7aa537f1f40c4f53abe5e8e8adec26dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:04:12.982970 2837 raft_consensus.cc:1261] T 8770d8ab6ba3446697c0ac8058f0ca2b P 867a225140b94ed584a79744322953b7 [term 1 FOLLOWER]: Refusing update from remote peer 7aa537f1f40c4f53abe5e8e8adec26dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:04:12.983289 3020 consensus_queue.cc:1035] T 8770d8ab6ba3446697c0ac8058f0ca2b P 7aa537f1f40c4f53abe5e8e8adec26dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3d81caeda554d0796d3454b7a08ba83" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:04:12.983513 3014 consensus_queue.cc:1035] T 8770d8ab6ba3446697c0ac8058f0ca2b P 7aa537f1f40c4f53abe5e8e8adec26dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "867a225140b94ed584a79744322953b7" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:04:12.985064 2890 heartbeater.cc:503] Master 127.25.130.61:40369 was elected leader, sending a full tablet report...
I20260812 19:04:12.985785 2985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.985440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.25.130.2:42433)
0812 19:04:12.985639 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:12.985704 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260812 19:04:12.986909 2986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.986613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.25.130.3:36975)
0812 19:04:12.986775 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:12.986835 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260812 19:04:12.989284 3019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.988829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.25.130.1:59421)
0812 19:04:12.989143 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:12.989205 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244}
I20260812 19:04:12.989939 2986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.989662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.25.130.3:45327)
0812 19:04:12.989816 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:12.989868 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260812 19:04:12.990913 3019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.990645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.25.130.1:58509)
0812 19:04:12.990786 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:12.990839 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 19:04:12.991391 2985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:12.991101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37435 (local address 127.25.130.2:37245)
0812 19:04:12.991266 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:12.991321 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260812 19:04:13.000201 2139 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:04:13.000372 2139 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:04:13.004407 2139 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 501fbfc27f1746b39f928f6228d3aa5a: Sending DeleteTablet for 3 replicas of tablet 8770d8ab6ba3446697c0ac8058f0ca2b
I20260812 19:04:13.005383 2648 tablet_service.cc:1514] Processing DeleteTablet for tablet 8770d8ab6ba3446697c0ac8058f0ca2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:04:12 UTC) from {username='oryx', principal='oryx/127.25.130.61@KRBTEST.COM'} at 127.0.0.1:37034
I20260812 19:04:13.005411 2488 tablet_service.cc:1514] Processing DeleteTablet for tablet 8770d8ab6ba3446697c0ac8058f0ca2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:04:12 UTC) from {username='oryx', principal='oryx/127.25.130.61@KRBTEST.COM'} at 127.0.0.1:46100
I20260812 19:04:13.005833 3034 tablet_replica.cc:330] T 8770d8ab6ba3446697c0ac8058f0ca2b P b3d81caeda554d0796d3454b7a08ba83: stopping tablet replica
I20260812 19:04:13.005798 2814 tablet_service.cc:1514] Processing DeleteTablet for tablet 8770d8ab6ba3446697c0ac8058f0ca2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:04:12 UTC) from {username='oryx', principal='oryx/127.25.130.61@KRBTEST.COM'} at 127.0.0.1:57776
I20260812 19:04:13.005952 3034 raft_consensus.cc:2229] T 8770d8ab6ba3446697c0ac8058f0ca2b P b3d81caeda554d0796d3454b7a08ba83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:04:13.006115 3034 raft_consensus.cc:2258] T 8770d8ab6ba3446697c0ac8058f0ca2b P b3d81caeda554d0796d3454b7a08ba83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:04:13.006271 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 2311
I20260812 19:04:13.008423 3034 ts_tablet_manager.cc:1905] T 8770d8ab6ba3446697c0ac8058f0ca2b P b3d81caeda554d0796d3454b7a08ba83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:04:13.010637 3036 tablet_replica.cc:330] T 8770d8ab6ba3446697c0ac8058f0ca2b P 867a225140b94ed584a79744322953b7: stopping tablet replica
I20260812 19:04:13.010726 3034 ts_tablet_manager.cc:1918] T 8770d8ab6ba3446697c0ac8058f0ca2b P b3d81caeda554d0796d3454b7a08ba83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:04:13.010793 3036 raft_consensus.cc:2229] T 8770d8ab6ba3446697c0ac8058f0ca2b P 867a225140b94ed584a79744322953b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:04:13.010814 3034 log.cc:1196] T 8770d8ab6ba3446697c0ac8058f0ca2b P b3d81caeda554d0796d3454b7a08ba83: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-1/wal/wals/8770d8ab6ba3446697c0ac8058f0ca2b
I20260812 19:04:13.010941 3036 raft_consensus.cc:2258] T 8770d8ab6ba3446697c0ac8058f0ca2b P 867a225140b94ed584a79744322953b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:04:13.011255 3034 ts_tablet_manager.cc:1939] T 8770d8ab6ba3446697c0ac8058f0ca2b P b3d81caeda554d0796d3454b7a08ba83: Deleting consensus metadata
I20260812 19:04:13.011698 3036 ts_tablet_manager.cc:1905] T 8770d8ab6ba3446697c0ac8058f0ca2b P 867a225140b94ed584a79744322953b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:04:13.011965 2122 catalog_manager.cc:4841] TS b3d81caeda554d0796d3454b7a08ba83 (127.25.130.2:41779): tablet 8770d8ab6ba3446697c0ac8058f0ca2b (table test-table [id=0ef585d4d3fe48968a48049272d3b628]) successfully deleted
I20260812 19:04:13.014477 3036 ts_tablet_manager.cc:1918] T 8770d8ab6ba3446697c0ac8058f0ca2b P 867a225140b94ed584a79744322953b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:04:13.014571 3036 log.cc:1196] T 8770d8ab6ba3446697c0ac8058f0ca2b P 867a225140b94ed584a79744322953b7: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561415988404-26120-0/minicluster-data/ts-2/wal/wals/8770d8ab6ba3446697c0ac8058f0ca2b
I20260812 19:04:13.014882 3036 ts_tablet_manager.cc:1939] T 8770d8ab6ba3446697c0ac8058f0ca2b P 867a225140b94ed584a79744322953b7: Deleting consensus metadata
I20260812 19:04:13.015645 2123 catalog_manager.cc:4841] TS 867a225140b94ed584a79744322953b7 (127.25.130.3:40061): tablet 8770d8ab6ba3446697c0ac8058f0ca2b (table test-table [id=0ef585d4d3fe48968a48049272d3b628]) successfully deleted
I20260812 19:04:13.017114 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 2571
I20260812 19:04:13.017395 3004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:13.017117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:37323 (local address 127.0.0.1:46118)
0812 19:04:13.017252 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:13.017305 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.1:37323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 19:04:13.022095 2121 catalog_manager.cc:4584] TS 7aa537f1f40c4f53abe5e8e8adec26dc (127.25.130.1:37323): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8770d8ab6ba3446697c0ac8058f0ca2b: Network error: Client connection negotiation failed: client connection to 127.25.130.1:37323: connect: Connection refused (error 111)
I20260812 19:04:13.026734 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 2733
I20260812 19:04:13.037027 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 1964
I20260812 19:04:13.047008 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 2098
2026-08-12T19:04:13Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7358 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:04:13 dist-test-slave-49h3 krb5kdc[3039](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:04:13 dist-test-slave-49h3 krb5kdc[3039](info): set up 2 sockets
Aug 12 19:04:13 dist-test-slave-49h3 krb5kdc[3039](info): commencing operation
krb5kdc: starting...
W20260812 19:04:14.108721 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:04:14 dist-test-slave-49h3 krb5kdc[3039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:04:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:04:14Z Disabled control of system clock
WARNING: no policy specified for oryx/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.62.keytab.
Entry for principal oryx/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.62.keytab.
I20260812 19:04:14.240233 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:44471
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.62.keytab
--principal=oryx/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:37901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:44471
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:04:14.355307 3054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:14.355626 3054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:14.355675 3054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:14.359800 3054 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:04:14.359874 3054 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 19:04:14.359891 3054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:14.359908 3054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:14.359925 3054 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:04:14.359941 3054 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:04:14.365577 3054 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:37901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:44471
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.62.keytab
--principal=oryx/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:44471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:14.366726 3054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:14.367748 3054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:14.376552 3061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:14.376658 3060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:14.376613 3064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:14.376732 3054 server_base.cc:1034] running on GCE node
I20260812 19:04:14.377429 3054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:14.378644 3054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:14.379812 3054 hybrid_clock.cc:648] HybridClock initialized: now 1786561454379798 us; error 30 us; skew 500 ppm
Aug 12 19:04:14 dist-test-slave-49h3 krb5kdc[3039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561454, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:14.382632 3054 init.cc:377] Logged in from keytab as oryx/127.25.130.62@KRBTEST.COM (short username oryx)
I20260812 19:04:14.383602 3054 webserver.cc:460] Webserver started at http://127.25.130.62:42309/ using document root <none> and password file <none>
I20260812 19:04:14.384183 3054 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:14.384255 3054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:14.384478 3054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:14.386739 3054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "b1beb356039e4f5185921f11c188d34b"
format_stamp: "Formatted at 2026-08-12 19:04:14 on dist-test-slave-49h3"
server_key: "cfe4049c19bb4735025602855629ecdd"
server_key_iv: "49aa00e7fff118ac119bb685c2077a8f"
server_key_version: "encryptionkey@0"
I20260812 19:04:14.387342 3054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "b1beb356039e4f5185921f11c188d34b"
format_stamp: "Formatted at 2026-08-12 19:04:14 on dist-test-slave-49h3"
server_key: "cfe4049c19bb4735025602855629ecdd"
server_key_iv: "49aa00e7fff118ac119bb685c2077a8f"
server_key_version: "encryptionkey@0"
I20260812 19:04:14.391431 3054 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 19:04:14.394064 3072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:14.395388 3054 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:04:14.395529 3054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "b1beb356039e4f5185921f11c188d34b"
format_stamp: "Formatted at 2026-08-12 19:04:14 on dist-test-slave-49h3"
server_key: "cfe4049c19bb4735025602855629ecdd"
server_key_iv: "49aa00e7fff118ac119bb685c2077a8f"
server_key_version: "encryptionkey@0"
I20260812 19:04:14.395642 3054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:14.408224 3054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:14.411496 3054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:14.411707 3054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:14.419867 3054 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:44471
I20260812 19:04:14.419884 3147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:44471 every 8 connection(s)
I20260812 19:04:14.420871 3054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:04:14.423869 3149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:14.427575 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 3054
I20260812 19:04:14.427699 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:04:14.427982 26120 external_mini_cluster.cc:1428] Setting key e5ce2eb633916d1f287c28af7c03c6f7
I20260812 19:04:14.430287 3149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1beb356039e4f5185921f11c188d34b: Bootstrap starting.
I20260812 19:04:14.433208 3149 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1beb356039e4f5185921f11c188d34b: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:14.434182 3149 log.cc:824] T 00000000000000000000000000000000 P b1beb356039e4f5185921f11c188d34b: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:14.436681 3149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1beb356039e4f5185921f11c188d34b: No bootstrap required, opened a new log
Aug 12 19:04:14 dist-test-slave-49h3 krb5kdc[3039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM
I20260812 19:04:14.441182 3149 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1beb356039e4f5185921f11c188d34b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1beb356039e4f5185921f11c188d34b" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 44471 } }
I20260812 19:04:14.441516 3149 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1beb356039e4f5185921f11c188d34b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:14.441591 3149 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1beb356039e4f5185921f11c188d34b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1beb356039e4f5185921f11c188d34b, State: Initialized, Role: FOLLOWER
I20260812 19:04:14.442133 3149 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1beb356039e4f5185921f11c188d34b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1beb356039e4f5185921f11c188d34b" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 44471 } }
I20260812 19:04:14.442328 3149 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1beb356039e4f5185921f11c188d34b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:04:14.442425 3149 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1beb356039e4f5185921f11c188d34b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:04:14.442536 3149 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1beb356039e4f5185921f11c188d34b [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:14.443828 3149 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1beb356039e4f5185921f11c188d34b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1beb356039e4f5185921f11c188d34b" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 44471 } }
I20260812 19:04:14.444155 3152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:14.429473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38618 (local address 127.25.130.62:44471)
0812 19:04:14.429954 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:14.429975 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:14.430020 (+ 45us) server_negotiation.cc:407] Connection header received
0812 19:04:14.430883 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:14.430909 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:14.431356 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:14.431750 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:14.432444 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:14.433978 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:14.434987 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:14.435695 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:14.435871 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:14.438540 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:14.438547 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:14.438566 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:14.438603 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:14.440527 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:14.440970 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:14.440976 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:14.440981 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:14.441150 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:14.441389 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:14.441392 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:14.441394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:14.441977 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:14.442196 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:14.442497 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 19:04:14.443000 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":132,"threads_started":1}
I20260812 19:04:14.444219 3149 leader_election.cc:302] T 00000000000000000000000000000000 P b1beb356039e4f5185921f11c188d34b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1beb356039e4f5185921f11c188d34b; no voters:
I20260812 19:04:14.444576 3149 leader_election.cc:288] T 00000000000000000000000000000000 P b1beb356039e4f5185921f11c188d34b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:04:14.444716 3155 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1beb356039e4f5185921f11c188d34b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:04:14.444947 3155 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1beb356039e4f5185921f11c188d34b [term 1 LEADER]: Becoming Leader. State: Replica: b1beb356039e4f5185921f11c188d34b, State: Running, Role: LEADER
I20260812 19:04:14.445292 3155 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1beb356039e4f5185921f11c188d34b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1beb356039e4f5185921f11c188d34b" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 44471 } }
I20260812 19:04:14.445901 3149 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1beb356039e4f5185921f11c188d34b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:04:14.447057 3156 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1beb356039e4f5185921f11c188d34b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1beb356039e4f5185921f11c188d34b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1beb356039e4f5185921f11c188d34b" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 44471 } } }
I20260812 19:04:14.447188 3156 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1beb356039e4f5185921f11c188d34b [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:14.447616 3165 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:04:14.447616 3157 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1beb356039e4f5185921f11c188d34b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1beb356039e4f5185921f11c188d34b. Latest consensus state: current_term: 1 leader_uuid: "b1beb356039e4f5185921f11c188d34b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1beb356039e4f5185921f11c188d34b" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 44471 } } }
I20260812 19:04:14.448333 3157 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1beb356039e4f5185921f11c188d34b [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:14.451208 3165 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:04:14.456792 3165 catalog_manager.cc:1347] Generated new cluster ID: c1773d300d5d41ed99cd3f806661d209
I20260812 19:04:14.456902 3165 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:04:14.465196 3165 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:04:14.466637 3165 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:04:14.483865 3165 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1beb356039e4f5185921f11c188d34b: Generated new TSK 0
I20260812 19:04:14.484676 3165 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.25.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.1.keytab.
Entry for principal oryx/127.25.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.1.keytab.
I20260812 19:04:14.521111 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:0
--local_ip_for_outbound_sockets=127.25.130.1
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:44471
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.1.keytab
--principal=oryx/127.25.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:37901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:04:14.642462 3178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:14.642741 3178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:14.642787 3178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:14.646931 3178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:14.647008 3178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:14.647049 3178 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 19:04:14.647087 3178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:04:14.652880 3178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:37901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.1.keytab
--principal=oryx/127.25.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.130.62:44471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:14.654101 3178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:14.655161 3178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:04:14.665663 3178 server_base.cc:1034] running on GCE node
W20260812 19:04:14.665602 3183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:14.665612 3184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:14.665604 3186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:14.666483 3178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:14.667183 3178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:14.668359 3178 hybrid_clock.cc:648] HybridClock initialized: now 1786561454668340 us; error 32 us; skew 500 ppm
Aug 12 19:04:14 dist-test-slave-49h3 krb5kdc[3039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561454, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:14.671278 3178 init.cc:377] Logged in from keytab as oryx/127.25.130.1@KRBTEST.COM (short username oryx)
I20260812 19:04:14.672323 3178 webserver.cc:460] Webserver started at http://127.25.130.1:42441/ using document root <none> and password file <none>
I20260812 19:04:14.672921 3178 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:14.672988 3178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:14.673198 3178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:14.675549 3178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-0/data/instance:
uuid: "d074369259f143bba8768daa96c9f43f"
format_stamp: "Formatted at 2026-08-12 19:04:14 on dist-test-slave-49h3"
server_key: "0c26151fbe94946575a6ddb339620e4a"
server_key_iv: "35c0cf7150109893214474111a613d74"
server_key_version: "encryptionkey@0"
I20260812 19:04:14.676203 3178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance:
uuid: "d074369259f143bba8768daa96c9f43f"
format_stamp: "Formatted at 2026-08-12 19:04:14 on dist-test-slave-49h3"
server_key: "0c26151fbe94946575a6ddb339620e4a"
server_key_iv: "35c0cf7150109893214474111a613d74"
server_key_version: "encryptionkey@0"
I20260812 19:04:14.680496 3178 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 19:04:14.683322 3194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:14.684767 3178 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 19:04:14.684916 3178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "d074369259f143bba8768daa96c9f43f"
format_stamp: "Formatted at 2026-08-12 19:04:14 on dist-test-slave-49h3"
server_key: "0c26151fbe94946575a6ddb339620e4a"
server_key_iv: "35c0cf7150109893214474111a613d74"
server_key_version: "encryptionkey@0"
I20260812 19:04:14.685050 3178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:14.703946 3178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:14.707232 3178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:14.707450 3178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:14.709098 3178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:14.709161 3178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:14.709220 3178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:14.709246 3178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 19:04:14 dist-test-slave-49h3 krb5kdc[3039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561454, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM
I20260812 19:04:14.720431 3178 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:37757
I20260812 19:04:14.720546 3327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:37757 every 8 connection(s)
I20260812 19:04:14.721616 3178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:04:14.723079 3152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:14.709981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:54935 (local address 127.25.130.62:44471)
0812 19:04:14.710151 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:14.710157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:14.711136 (+ 979us) server_negotiation.cc:407] Connection header received
0812 19:04:14.712426 (+ 1290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:14.712430 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:14.712496 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:14.712588 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:14.713990 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:14.714564 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:14.715873 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:14.716132 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:14.716227 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:14.719050 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:14.719053 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:14.719055 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:14.719085 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:14.720582 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:14.721222 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:14.721225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:14.721227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:14.721312 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:14.721774 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:14.721777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:14.721778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:14.722002 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:14.722106 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:14.722746 (+ 640us) server_negotiation.cc:299] Negotiation successful
0812 19:04:14.722901 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 19:04:14.723852 3204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:14.710386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:44471 (local address 127.25.130.1:54935)
0812 19:04:14.710933 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:14.710999 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:14.712120 (+ 1121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:14.712824 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:14.712831 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:14.713221 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:14.713828 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:14.713846 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:14.714703 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:14.714706 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:14.715735 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:14.715748 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:14.716310 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:14.716314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:14.716525 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:14.717144 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:14.717163 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:14.718840 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:14.720727 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:14.720734 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:14.720749 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:14.721113 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:14.721400 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:14.721403 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:14.721405 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:14.721637 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:14.722164 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:14.722170 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:14.722471 (+ 301us) client_negotiation.cc:769] Sending connection context
0812 19:04:14.722660 (+ 189us) client_negotiation.cc:240] Negotiation successful
0812 19:04:14.723011 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":140,"threads_started":1}
I20260812 19:04:14.725076 3328 heartbeater.cc:347] Connected to a master server at 127.25.130.62:44471
I20260812 19:04:14.725230 3328 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:14.725795 3328 heartbeater.cc:511] Master 127.25.130.62:44471 requested a full tablet report, sending...
I20260812 19:04:14.727281 3094 ts_manager.cc:194] Registered new tserver with Master: d074369259f143bba8768daa96c9f43f (127.25.130.1:37757)
I20260812 19:04:14.728482 3094 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.130.1@KRBTEST.COM'} at 127.25.130.1:54935
I20260812 19:04:14.730110 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 3178
I20260812 19:04:14.730254 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance
I20260812 19:04:14.730584 26120 external_mini_cluster.cc:1428] Setting key 260c3f3594bebe4f5f8cf79913482460
I20260812 19:04:14.734700 3334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:14.727258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:44471 (local address 127.25.130.1:56711)
0812 19:04:14.727587 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:14.727600 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:14.727703 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:14.727921 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:14.727924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:14.728133 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:14.728239 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:14.728246 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:14.729034 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:14.729037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:14.729689 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:14.729699 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:14.730136 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:14.730140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:14.730277 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:14.730795 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:14.730810 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:14.731255 (+ 445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:14.733029 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:14.733033 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:14.733035 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:14.733261 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:14.733589 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:14.733593 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:14.733595 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:14.733696 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:14.734161 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:14.734164 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:14.734271 (+ 107us) client_negotiation.cc:769] Sending connection context
0812 19:04:14.734368 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 19:04:14.734510 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":91,"threads_started":1}
I20260812 19:04:14.734757 3152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:14.727332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:56711 (local address 127.25.130.62:44471)
0812 19:04:14.727466 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:14.727471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:14.727666 (+ 195us) server_negotiation.cc:407] Connection header received
0812 19:04:14.727780 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:14.727784 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:14.727827 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:14.727890 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:14.728350 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:14.728911 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:14.729801 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:14.730027 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:14.730117 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:14.731383 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:14.731386 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:14.731388 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:14.731412 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:14.732911 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:14.733361 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:14.733365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:14.733366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:14.733478 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:14.733785 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:14.733789 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:14.733791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:14.734044 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:14.734162 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:14.734440 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 19:04:14.734579 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
WARNING: no policy specified for oryx/127.25.130.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.130.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.2.keytab.
Entry for principal oryx/127.25.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.2.keytab.
I20260812 19:04:14.761144 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:0
--local_ip_for_outbound_sockets=127.25.130.2
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:44471
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.2.keytab
--principal=oryx/127.25.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:37901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:04:14.877583 3338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:14.877878 3338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:14.877926 3338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:14.882078 3338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:14.882154 3338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:14.882197 3338 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 19:04:14.882229 3338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:04:14.887723 3338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:37901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.2.keytab
--principal=oryx/127.25.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.130.62:44471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:14.888893 3338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:14.890026 3338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:14.899927 3348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:14.899971 3349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:14.899932 3351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:14.900162 3338 server_base.cc:1034] running on GCE node
I20260812 19:04:14.900729 3338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:14.901495 3338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:14.902680 3338 hybrid_clock.cc:648] HybridClock initialized: now 1786561454902656 us; error 35 us; skew 500 ppm
Aug 12 19:04:14 dist-test-slave-49h3 krb5kdc[3039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561454, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:14.905604 3338 init.cc:377] Logged in from keytab as oryx/127.25.130.2@KRBTEST.COM (short username oryx)
I20260812 19:04:14.906585 3338 webserver.cc:460] Webserver started at http://127.25.130.2:33607/ using document root <none> and password file <none>
I20260812 19:04:14.907140 3338 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:14.907258 3338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:14.907462 3338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:14.909696 3338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-1/data/instance:
uuid: "0f0aa9470f4b4e3aac661d0089845b9a"
format_stamp: "Formatted at 2026-08-12 19:04:14 on dist-test-slave-49h3"
server_key: "b7c0c031b5da4af8ae3af8d3cd349ca8"
server_key_iv: "d9f39a943c8e5c8cb6be50888052e457"
server_key_version: "encryptionkey@0"
I20260812 19:04:14.910254 3338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance:
uuid: "0f0aa9470f4b4e3aac661d0089845b9a"
format_stamp: "Formatted at 2026-08-12 19:04:14 on dist-test-slave-49h3"
server_key: "b7c0c031b5da4af8ae3af8d3cd349ca8"
server_key_iv: "d9f39a943c8e5c8cb6be50888052e457"
server_key_version: "encryptionkey@0"
I20260812 19:04:14.914427 3338 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:04:14.917016 3361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:14.918473 3338 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:04:14.918604 3338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "0f0aa9470f4b4e3aac661d0089845b9a"
format_stamp: "Formatted at 2026-08-12 19:04:14 on dist-test-slave-49h3"
server_key: "b7c0c031b5da4af8ae3af8d3cd349ca8"
server_key_iv: "d9f39a943c8e5c8cb6be50888052e457"
server_key_version: "encryptionkey@0"
I20260812 19:04:14.918725 3338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:14.941598 3338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:14.944770 3338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:14.944979 3338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:14.946717 3338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:14.946786 3338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:14.946851 3338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:14.946880 3338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 19:04:14 dist-test-slave-49h3 krb5kdc[3039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561454, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM
I20260812 19:04:14.957571 3338 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:33077
I20260812 19:04:14.957638 3494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:33077 every 8 connection(s)
I20260812 19:04:14.958613 3338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:04:14.959105 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 3338
I20260812 19:04:14.959227 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance
I20260812 19:04:14.959520 26120 external_mini_cluster.cc:1428] Setting key 9deaea1b9ff060d28410d2f9e71eb682
I20260812 19:04:14.960973 3152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:14.947664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:33377 (local address 127.25.130.62:44471)
0812 19:04:14.947835 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:14.947840 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:14.948803 (+ 963us) server_negotiation.cc:407] Connection header received
0812 19:04:14.950049 (+ 1246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:14.950054 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:14.950123 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:14.950235 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:14.951601 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:14.952147 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:14.953956 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:14.954296 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:14.954413 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:14.957164 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:14.957168 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:14.957171 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:14.957202 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:14.958660 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:14.959320 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:14.959324 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:14.959325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:14.959431 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:14.959807 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:14.959810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:14.959812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:14.960074 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:14.960176 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:14.960682 (+ 506us) server_negotiation.cc:299] Negotiation successful
0812 19:04:14.960818 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 19:04:14.961874 3370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:14.948143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:44471 (local address 127.25.130.2:33377)
0812 19:04:14.948616 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:14.948693 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:14.949760 (+ 1067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:14.950432 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:14.950442 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:14.950854 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:14.951425 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:14.951441 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:14.952268 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:14.952272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:14.953801 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:14.953816 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:14.954472 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:14.954477 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:14.954656 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:14.955329 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:14.955347 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:14.956946 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:14.958816 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:14.958823 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:14.958839 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:14.959216 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:14.959532 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:14.959535 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:14.959537 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:14.959721 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:14.960180 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:14.960187 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:14.960444 (+ 257us) client_negotiation.cc:769] Sending connection context
0812 19:04:14.960616 (+ 172us) client_negotiation.cc:240] Negotiation successful
0812 19:04:14.960968 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":110,"threads_started":1}
I20260812 19:04:14.963277 3495 heartbeater.cc:347] Connected to a master server at 127.25.130.62:44471
I20260812 19:04:14.963439 3495 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:14.963953 3495 heartbeater.cc:511] Master 127.25.130.62:44471 requested a full tablet report, sending...
I20260812 19:04:14.965104 3093 ts_manager.cc:194] Registered new tserver with Master: 0f0aa9470f4b4e3aac661d0089845b9a (127.25.130.2:33077)
I20260812 19:04:14.965896 3093 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.130.2@KRBTEST.COM'} at 127.25.130.2:33377
WARNING: no policy specified for oryx/127.25.130.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.130.3@KRBTEST.COM" created.
I20260812 19:04:14.975158 3152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:14.965710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:53611 (local address 127.25.130.62:44471)
0812 19:04:14.965844 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:14.965848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:14.967632 (+ 1784us) server_negotiation.cc:407] Connection header received
0812 19:04:14.967750 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:14.967754 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:14.967803 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:14.967884 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:14.968452 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:14.968982 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:14.970172 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:14.970406 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:14.970490 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:14.971743 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:14.971748 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:14.971750 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:14.971779 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:14.973151 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:14.973679 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:14.973683 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:14.973684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:14.973763 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:14.974128 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:14.974132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:14.974134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:14.974426 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:14.974529 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:14.974858 (+ 329us) server_negotiation.cc:299] Negotiation successful
0812 19:04:14.974980 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 19:04:14.975178 3504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:14.965660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:44471 (local address 127.25.130.2:53611)
0812 19:04:14.967515 (+ 1855us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:14.967531 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:14.967659 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:14.967919 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:14.967923 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:14.968158 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:14.968301 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:14.968309 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:14.969113 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:14.969117 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:14.970045 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:14.970058 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:14.970526 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:14.970530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:14.970636 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:14.971146 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:14.971162 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:14.971587 (+ 425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:14.973278 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:14.973281 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:14.973283 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:14.973579 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:14.973894 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:14.973897 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:14.973899 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:14.974007 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:14.974543 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:14.974546 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:14.974645 (+ 99us) client_negotiation.cc:769] Sending connection context
0812 19:04:14.974789 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 19:04:14.974961 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1748,"thread_start_us":119,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.3.keytab.
Entry for principal oryx/127.25.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.3.keytab.
I20260812 19:04:14.992920 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:0
--local_ip_for_outbound_sockets=127.25.130.3
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:44471
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.3.keytab
--principal=oryx/127.25.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:37901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:04:15.111447 3506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:15.111766 3506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:15.111814 3506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:15.116039 3506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:15.116128 3506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:15.116171 3506 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 19:04:15.116204 3506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:04:15.121848 3506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:37901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.3.keytab
--principal=oryx/127.25.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.130.62:44471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:15.123054 3506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:15.124141 3506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:04:15.134692 3506 server_base.cc:1034] running on GCE node
W20260812 19:04:15.134678 3512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:15.134677 3515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:15.134877 3517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:15.135306 3506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:15.136055 3506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:15.137251 3506 hybrid_clock.cc:648] HybridClock initialized: now 1786561455137218 us; error 44 us; skew 500 ppm
Aug 12 19:04:15 dist-test-slave-49h3 krb5kdc[3039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561455, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:15.140224 3506 init.cc:377] Logged in from keytab as oryx/127.25.130.3@KRBTEST.COM (short username oryx)
I20260812 19:04:15.141187 3506 webserver.cc:460] Webserver started at http://127.25.130.3:42885/ using document root <none> and password file <none>
I20260812 19:04:15.141846 3506 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:15.141925 3506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:15.142163 3506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:15.144397 3506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-2/data/instance:
uuid: "6183cb8c293e439daea1a4221ba83794"
format_stamp: "Formatted at 2026-08-12 19:04:15 on dist-test-slave-49h3"
server_key: "e9247b4b6fd3ae1a22b220ffd62240bd"
server_key_iv: "38fc19445cb866e17d1c0c94396ff075"
server_key_version: "encryptionkey@0"
I20260812 19:04:15.144997 3506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance:
uuid: "6183cb8c293e439daea1a4221ba83794"
format_stamp: "Formatted at 2026-08-12 19:04:15 on dist-test-slave-49h3"
server_key: "e9247b4b6fd3ae1a22b220ffd62240bd"
server_key_iv: "38fc19445cb866e17d1c0c94396ff075"
server_key_version: "encryptionkey@0"
I20260812 19:04:15.149343 3506 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 19:04:15.152024 3525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:15.153339 3506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:04:15.153486 3506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "6183cb8c293e439daea1a4221ba83794"
format_stamp: "Formatted at 2026-08-12 19:04:15 on dist-test-slave-49h3"
server_key: "e9247b4b6fd3ae1a22b220ffd62240bd"
server_key_iv: "38fc19445cb866e17d1c0c94396ff075"
server_key_version: "encryptionkey@0"
I20260812 19:04:15.153604 3506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:15.178853 3506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:15.182204 3506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:15.182399 3506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:15.184327 3506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:15.184397 3506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:15.184453 3506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:15.184484 3506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:15.194166 3506 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:46379
Aug 12 19:04:15 dist-test-slave-49h3 krb5kdc[3039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561455, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM
I20260812 19:04:15.194245 3658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:46379 every 8 connection(s)
I20260812 19:04:15.195108 3506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:04:15.198565 3152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.185121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:46977 (local address 127.25.130.62:44471)
0812 19:04:15.185298 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:15.185304 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:15.186232 (+ 928us) server_negotiation.cc:407] Connection header received
0812 19:04:15.187362 (+ 1130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:15.187367 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:15.187429 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:15.187488 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:15.189341 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.189956 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.191453 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.191683 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.191777 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.194863 (+ 3086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:15.194868 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:15.194871 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:15.194899 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:15.196497 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:15.197000 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:15.197004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:15.197006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:15.197093 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:15.197422 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:15.197425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:15.197427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:15.197662 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:15.197758 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:15.198267 (+ 509us) server_negotiation.cc:299] Negotiation successful
0812 19:04:15.198398 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 19:04:15.199450 3534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.185566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:44471 (local address 127.25.130.3:46977)
0812 19:04:15.186063 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:15.186116 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:15.187076 (+ 960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:15.187963 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:15.187970 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:15.188436 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:15.189184 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.189207 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.190094 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.190098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.191312 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.191325 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.191836 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.191840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.192045 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.192779 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:15.192806 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:15.194671 (+ 1865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:15.196622 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:15.196628 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:15.196643 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:15.196899 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:15.197178 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:15.197181 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:15.197183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:15.197334 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:15.197772 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:15.197779 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:15.198027 (+ 248us) client_negotiation.cc:769] Sending connection context
0812 19:04:15.198207 (+ 180us) client_negotiation.cc:240] Negotiation successful
0812 19:04:15.198541 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":122,"threads_started":1}
I20260812 19:04:15.200668 3659 heartbeater.cc:347] Connected to a master server at 127.25.130.62:44471
I20260812 19:04:15.200825 3659 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:15.201181 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 3506
I20260812 19:04:15.201316 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance
I20260812 19:04:15.201473 3659 heartbeater.cc:511] Master 127.25.130.62:44471 requested a full tablet report, sending...
I20260812 19:04:15.201706 26120 external_mini_cluster.cc:1428] Setting key c30e516145f9843008980ad5fc086a97
I20260812 19:04:15.202658 3093 ts_manager.cc:194] Registered new tserver with Master: 6183cb8c293e439daea1a4221ba83794 (127.25.130.3:46379)
I20260812 19:04:15.203447 3093 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.130.3@KRBTEST.COM'} at 127.25.130.3:46977
I20260812 19:04:15.204444 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:04:15.212821 3152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.203221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:43955 (local address 127.25.130.62:44471)
0812 19:04:15.203339 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:15.203343 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:15.203513 (+ 170us) server_negotiation.cc:407] Connection header received
0812 19:04:15.203810 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:15.203813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:15.203862 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:15.203940 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:15.206053 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.206830 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.208062 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.208287 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.208373 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.209570 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:15.209573 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:15.209576 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:15.209606 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:15.211158 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:15.211689 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:15.211691 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:15.211692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:15.211772 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:15.212063 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:15.212065 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:15.212066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:15.212241 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:15.212312 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:15.212586 (+ 274us) server_negotiation.cc:299] Negotiation successful
0812 19:04:15.212693 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 19:04:15.213177 3666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.203150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:44471 (local address 127.25.130.3:43955)
0812 19:04:15.203439 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:15.203451 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:15.203588 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:15.205581 (+ 1993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:15.205586 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:15.205792 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:15.205899 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.205905 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.207012 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.207016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.207887 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.207901 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.208410 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.208414 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.208501 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.209047 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:15.209060 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:15.209404 (+ 344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:15.211296 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:15.211300 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:15.211302 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:15.211580 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:15.211851 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:15.211854 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:15.211856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:15.211963 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:15.212323 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:15.212325 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:15.212431 (+ 106us) client_negotiation.cc:769] Sending connection context
0812 19:04:15.212533 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 19:04:15.212700 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":110,"threads_started":1}
Aug 12 19:04:15 dist-test-slave-49h3 krb5kdc[3039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561455, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 19:04:15 dist-test-slave-49h3 krb5kdc[3039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:15 dist-test-slave-49h3 krb5kdc[3039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
W20260812 19:04:15.231925 3152 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:15.225517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38620 (local address 127.25.130.62:44471)
0812 19:04:15.225639 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:15.225642 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:15.225657 (+ 15us) server_negotiation.cc:407] Connection header received
0812 19:04:15.225705 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:15.225707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:15.225750 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:15.225825 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:15.226631 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.227181 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.228068 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.228309 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.228403 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.231758 (+ 3355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38620: BlockingRecv error: recv got EOF from 127.0.0.1:38620 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 19:04:15 dist-test-slave-49h3 krb5kdc[3039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561455, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM
I20260812 19:04:15.241479 3152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.233536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38622 (local address 127.25.130.62:44471)
0812 19:04:15.233655 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:15.233658 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:15.233792 (+ 134us) server_negotiation.cc:407] Connection header received
0812 19:04:15.233903 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:15.233905 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:15.233947 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:15.234017 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:15.234658 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.235224 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.236045 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.236281 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.236370 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.238809 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:15.238813 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:15.238815 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:15.238845 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:15.240163 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:15.240522 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:15.240524 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:15.240525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:15.240597 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:15.240779 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:15.240781 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:15.240782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:15.240955 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:15.241056 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:15.241225 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 19:04:15.241323 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 19:04:15.244530 3094 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38622:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:04:15.247812 3094 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:04:15.262437 3689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.256680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:46379 (local address 127.0.0.1:41036)
0812 19:04:15.257379 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:15.257393 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:15.257551 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:15.258177 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:15.258179 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:15.258197 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:15.258304 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.258311 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.259688 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.259691 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.260796 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.260804 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.261846 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.261850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.261984 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.262034 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 19:04:15.262143 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 19:04:15.262278 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":93,"threads_started":1}
I20260812 19:04:15.262598 3691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.257678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60240 (local address 127.25.130.1:37757)
0812 19:04:15.258453 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:15.258460 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:15.258482 (+ 22us) server_negotiation.cc:407] Connection header received
0812 19:04:15.258566 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:15.258571 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:15.258842 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:15.258986 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:15.259216 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.260094 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.261263 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.261992 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.262119 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.262264 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:15.262373 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 19:04:15.262475 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":96,"threads_started":1}
I20260812 19:04:15.262655 3688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.257114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:37757 (local address 127.0.0.1:60240)
0812 19:04:15.257887 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:15.257900 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:15.257988 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:15.258985 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:15.258989 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:15.259010 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:15.259103 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.259109 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.260211 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.260214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.261124 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.261134 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.262115 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.262118 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.262245 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.262272 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 19:04:15.262345 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 19:04:15.262420 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"mutex_wait_us":20,"spinlock_wait_cycles":1280,"thread_start_us":104,"threads_started":1}
I20260812 19:04:15.263227 3686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.256344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:33077 (local address 127.0.0.1:44490)
0812 19:04:15.256715 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:15.256776 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:15.256904 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:15.257938 (+ 1034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:15.257946 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:15.257988 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:15.258157 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.258166 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.259352 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.259355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.260678 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.260689 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.262082 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.262086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.262955 (+ 869us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.262985 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 19:04:15.263054 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 19:04:15.263121 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":326,"spinlock_wait_cycles":6400,"thread_start_us":102,"threads_started":1}
I20260812 19:04:15.263600 3687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.256531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44490 (local address 127.25.130.2:33077)
0812 19:04:15.256948 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:15.256961 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:15.256987 (+ 26us) server_negotiation.cc:407] Connection header received
0812 19:04:15.257193 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:15.257200 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:15.257422 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:15.257608 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:15.258296 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.259260 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.260831 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.261725 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.261827 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.262001 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:15.263391 (+ 1390us) server_negotiation.cc:299] Negotiation successful
0812 19:04:15.263488 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":101,"threads_started":1}
I20260812 19:04:15.264488 3690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.257183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41036 (local address 127.25.130.3:46379)
0812 19:04:15.257721 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:15.257732 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:15.257754 (+ 22us) server_negotiation.cc:407] Connection header received
0812 19:04:15.257829 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:15.257834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:15.258042 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:15.258193 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:15.258399 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.259564 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.260918 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.261711 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.261827 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.262002 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:15.262247 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 19:04:15.264352 (+ 2105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":158,"threads_started":1}
I20260812 19:04:15.266089 3415 tablet_service.cc:1467] Processing CreateTablet for tablet 6b67f33b58f44c66a3745162234cb13d (DEFAULT_TABLE table=test-table [id=169f7a2ecaf44bb0a41d44139d2b1e44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:04:15.265913 3250 tablet_service.cc:1467] Processing CreateTablet for tablet 6b67f33b58f44c66a3745162234cb13d (DEFAULT_TABLE table=test-table [id=169f7a2ecaf44bb0a41d44139d2b1e44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:04:15.267050 3415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b67f33b58f44c66a3745162234cb13d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:15.267081 3250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b67f33b58f44c66a3745162234cb13d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:15.267071 3581 tablet_service.cc:1467] Processing CreateTablet for tablet 6b67f33b58f44c66a3745162234cb13d (DEFAULT_TABLE table=test-table [id=169f7a2ecaf44bb0a41d44139d2b1e44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:04:15.268078 3581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b67f33b58f44c66a3745162234cb13d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:15.274824 3693 tablet_bootstrap.cc:492] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f: Bootstrap starting.
I20260812 19:04:15.275255 3692 tablet_bootstrap.cc:492] T 6b67f33b58f44c66a3745162234cb13d P 0f0aa9470f4b4e3aac661d0089845b9a: Bootstrap starting.
I20260812 19:04:15.275800 3694 tablet_bootstrap.cc:492] T 6b67f33b58f44c66a3745162234cb13d P 6183cb8c293e439daea1a4221ba83794: Bootstrap starting.
I20260812 19:04:15.277237 3693 tablet_bootstrap.cc:654] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:15.277398 3692 tablet_bootstrap.cc:654] T 6b67f33b58f44c66a3745162234cb13d P 0f0aa9470f4b4e3aac661d0089845b9a: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:15.277947 3694 tablet_bootstrap.cc:654] T 6b67f33b58f44c66a3745162234cb13d P 6183cb8c293e439daea1a4221ba83794: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:15.278057 3693 log.cc:824] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:15.278277 3692 log.cc:824] T 6b67f33b58f44c66a3745162234cb13d P 0f0aa9470f4b4e3aac661d0089845b9a: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:15.278726 3694 log.cc:824] T 6b67f33b58f44c66a3745162234cb13d P 6183cb8c293e439daea1a4221ba83794: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:15.280193 3693 tablet_bootstrap.cc:492] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f: No bootstrap required, opened a new log
I20260812 19:04:15.280267 3692 tablet_bootstrap.cc:492] T 6b67f33b58f44c66a3745162234cb13d P 0f0aa9470f4b4e3aac661d0089845b9a: No bootstrap required, opened a new log
I20260812 19:04:15.280398 3693 ts_tablet_manager.cc:1397] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 19:04:15.280460 3692 ts_tablet_manager.cc:1397] T 6b67f33b58f44c66a3745162234cb13d P 0f0aa9470f4b4e3aac661d0089845b9a: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260812 19:04:15.283308 3694 tablet_bootstrap.cc:492] T 6b67f33b58f44c66a3745162234cb13d P 6183cb8c293e439daea1a4221ba83794: No bootstrap required, opened a new log
I20260812 19:04:15.283524 3694 ts_tablet_manager.cc:1397] T 6b67f33b58f44c66a3745162234cb13d P 6183cb8c293e439daea1a4221ba83794: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260812 19:04:15.283565 3692 raft_consensus.cc:348] T 6b67f33b58f44c66a3745162234cb13d P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.283828 3692 raft_consensus.cc:374] T 6b67f33b58f44c66a3745162234cb13d P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:15.283881 3692 raft_consensus.cc:729] T 6b67f33b58f44c66a3745162234cb13d P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f0aa9470f4b4e3aac661d0089845b9a, State: Initialized, Role: FOLLOWER
I20260812 19:04:15.284425 3692 consensus_queue.cc:260] T 6b67f33b58f44c66a3745162234cb13d P 0f0aa9470f4b4e3aac661d0089845b9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.284682 3693 raft_consensus.cc:348] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.284955 3693 raft_consensus.cc:374] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:15.285012 3693 raft_consensus.cc:729] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d074369259f143bba8768daa96c9f43f, State: Initialized, Role: FOLLOWER
I20260812 19:04:15.285313 3495 heartbeater.cc:503] Master 127.25.130.62:44471 was elected leader, sending a full tablet report...
I20260812 19:04:15.285475 3693 consensus_queue.cc:260] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.285578 3692 ts_tablet_manager.cc:1428] T 6b67f33b58f44c66a3745162234cb13d P 0f0aa9470f4b4e3aac661d0089845b9a: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260812 19:04:15.286110 3328 heartbeater.cc:503] Master 127.25.130.62:44471 was elected leader, sending a full tablet report...
I20260812 19:04:15.286468 3693 ts_tablet_manager.cc:1428] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260812 19:04:15.288740 3694 raft_consensus.cc:348] T 6b67f33b58f44c66a3745162234cb13d P 6183cb8c293e439daea1a4221ba83794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.289013 3694 raft_consensus.cc:374] T 6b67f33b58f44c66a3745162234cb13d P 6183cb8c293e439daea1a4221ba83794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:15.289078 3694 raft_consensus.cc:729] T 6b67f33b58f44c66a3745162234cb13d P 6183cb8c293e439daea1a4221ba83794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6183cb8c293e439daea1a4221ba83794, State: Initialized, Role: FOLLOWER
I20260812 19:04:15.289556 3694 consensus_queue.cc:260] T 6b67f33b58f44c66a3745162234cb13d P 6183cb8c293e439daea1a4221ba83794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.290289 3659 heartbeater.cc:503] Master 127.25.130.62:44471 was elected leader, sending a full tablet report...
I20260812 19:04:15.290565 3694 ts_tablet_manager.cc:1428] T 6b67f33b58f44c66a3745162234cb13d P 6183cb8c293e439daea1a4221ba83794: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 19:04:15.319993 3700 raft_consensus.cc:482] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:04:15.320206 3700 raft_consensus.cc:504] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.321322 3700 leader_election.cc:288] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f0aa9470f4b4e3aac661d0089845b9a (127.25.130.2:33077), 6183cb8c293e439daea1a4221ba83794 (127.25.130.3:46379)
I20260812 19:04:15.325934 3687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.321713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:59239 (local address 127.25.130.2:33077)
0812 19:04:15.321853 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:15.321857 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:15.322174 (+ 317us) server_negotiation.cc:407] Connection header received
0812 19:04:15.322295 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:15.322299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:15.322352 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:15.322434 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:15.322816 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.323633 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.324890 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.325312 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.325421 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.325500 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:15.325724 (+ 224us) server_negotiation.cc:299] Negotiation successful
0812 19:04:15.325804 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 19:04:15.325973 3690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.321825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:47097 (local address 127.25.130.3:46379)
0812 19:04:15.321976 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:15.321980 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:15.322146 (+ 166us) server_negotiation.cc:407] Connection header received
0812 19:04:15.322321 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:15.322324 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:15.322367 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:15.322436 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:15.322816 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.323620 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.324894 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.325311 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.325463 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.325516 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:15.325793 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 19:04:15.325870 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 19:04:15.325961 3702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.321592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:33077 (local address 127.25.130.1:59239)
0812 19:04:15.322076 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:15.322088 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:15.322196 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:15.322456 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:15.322459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:15.322479 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:15.322706 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.322714 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.323742 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.323745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.324761 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.324773 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.325488 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.325492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.325603 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.325624 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 19:04:15.325686 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 19:04:15.325751 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"mutex_wait_us":239,"spinlock_wait_cycles":33664,"thread_start_us":102,"threads_started":1}
I20260812 19:04:15.325976 3703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.321592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:46379 (local address 127.25.130.1:47097)
0812 19:04:15.322041 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:15.322065 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:15.322230 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:15.322503 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:15.322506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:15.322533 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:15.322712 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.322717 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.323742 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.323746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.324762 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.324776 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.325571 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.325574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.325658 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.325680 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 19:04:15.325757 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 19:04:15.325823 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":114,"spinlock_wait_cycles":20864,"thread_start_us":143,"threads_started":1}
I20260812 19:04:15.326915 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b67f33b58f44c66a3745162234cb13d" candidate_uuid: "d074369259f143bba8768daa96c9f43f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" is_pre_election: true
I20260812 19:04:15.327039 3606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b67f33b58f44c66a3745162234cb13d" candidate_uuid: "d074369259f143bba8768daa96c9f43f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6183cb8c293e439daea1a4221ba83794" is_pre_election: true
I20260812 19:04:15.327239 3441 raft_consensus.cc:2454] T 6b67f33b58f44c66a3745162234cb13d P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d074369259f143bba8768daa96c9f43f in term 0.
I20260812 19:04:15.327323 3606 raft_consensus.cc:2454] T 6b67f33b58f44c66a3745162234cb13d P 6183cb8c293e439daea1a4221ba83794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d074369259f143bba8768daa96c9f43f in term 0.
I20260812 19:04:15.327777 3197 leader_election.cc:302] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f0aa9470f4b4e3aac661d0089845b9a, d074369259f143bba8768daa96c9f43f; no voters:
I20260812 19:04:15.328063 3700 raft_consensus.cc:2781] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:04:15.328143 3700 raft_consensus.cc:482] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:04:15.328173 3700 raft_consensus.cc:3037] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:15.329224 3700 raft_consensus.cc:504] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.329599 3700 leader_election.cc:288] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f [CANDIDATE]: Term 1 election: Requested vote from peers 0f0aa9470f4b4e3aac661d0089845b9a (127.25.130.2:33077), 6183cb8c293e439daea1a4221ba83794 (127.25.130.3:46379)
I20260812 19:04:15.329962 3606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b67f33b58f44c66a3745162234cb13d" candidate_uuid: "d074369259f143bba8768daa96c9f43f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6183cb8c293e439daea1a4221ba83794"
I20260812 19:04:15.330066 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b67f33b58f44c66a3745162234cb13d" candidate_uuid: "d074369259f143bba8768daa96c9f43f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f0aa9470f4b4e3aac661d0089845b9a"
I20260812 19:04:15.330086 3606 raft_consensus.cc:3037] T 6b67f33b58f44c66a3745162234cb13d P 6183cb8c293e439daea1a4221ba83794 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:15.330202 3441 raft_consensus.cc:3037] T 6b67f33b58f44c66a3745162234cb13d P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:15.331228 3441 raft_consensus.cc:2454] T 6b67f33b58f44c66a3745162234cb13d P 0f0aa9470f4b4e3aac661d0089845b9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d074369259f143bba8768daa96c9f43f in term 1.
I20260812 19:04:15.331543 3197 leader_election.cc:302] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f0aa9470f4b4e3aac661d0089845b9a, d074369259f143bba8768daa96c9f43f; no voters:
I20260812 19:04:15.331622 3606 raft_consensus.cc:2454] T 6b67f33b58f44c66a3745162234cb13d P 6183cb8c293e439daea1a4221ba83794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d074369259f143bba8768daa96c9f43f in term 1.
I20260812 19:04:15.331768 3700 raft_consensus.cc:2781] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:04:15.332029 3700 raft_consensus.cc:686] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f [term 1 LEADER]: Becoming Leader. State: Replica: d074369259f143bba8768daa96c9f43f, State: Running, Role: LEADER
I20260812 19:04:15.332417 3700 consensus_queue.cc:237] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.335712 3094 catalog_manager.cc:5495] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f reported cstate change: term changed from 0 to 1, leader changed from <none> to d074369259f143bba8768daa96c9f43f (127.25.130.1). New cstate: current_term: 1 leader_uuid: "d074369259f143bba8768daa96c9f43f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } health_report { overall_health: UNKNOWN } } }
I20260812 19:04:15.363994 3691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.360248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60246 (local address 127.25.130.1:37757)
0812 19:04:15.360396 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:15.360401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:15.360417 (+ 16us) server_negotiation.cc:407] Connection header received
0812 19:04:15.360536 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:15.360540 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:15.360610 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:15.360706 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:04:15.360957 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.361506 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.362376 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.362612 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.362705 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.363023 (+ 318us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:04:15.363545 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:04:15.363664 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:15.363771 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 19:04:15.363844 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 19:04:15.370242 3606 raft_consensus.cc:1261] T 6b67f33b58f44c66a3745162234cb13d P 6183cb8c293e439daea1a4221ba83794 [term 1 FOLLOWER]: Refusing update from remote peer d074369259f143bba8768daa96c9f43f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:04:15.370484 3441 raft_consensus.cc:1261] T 6b67f33b58f44c66a3745162234cb13d P 0f0aa9470f4b4e3aac661d0089845b9a [term 1 FOLLOWER]: Refusing update from remote peer d074369259f143bba8768daa96c9f43f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:04:15.370985 3704 consensus_queue.cc:1035] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:04:15.371217 3700 consensus_queue.cc:1035] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:04:15.388545 3094 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:04:15.388767 3094 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:04:15.391404 3094 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b1beb356039e4f5185921f11c188d34b: Sending DeleteTablet for 3 replicas of tablet 6b67f33b58f44c66a3745162234cb13d
I20260812 19:04:15.392372 3581 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b67f33b58f44c66a3745162234cb13d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:04:15 UTC) from {username='oryx', principal='oryx/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:41036
I20260812 19:04:15.392443 3250 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b67f33b58f44c66a3745162234cb13d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:04:15 UTC) from {username='oryx', principal='oryx/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:60240
I20260812 19:04:15.392951 3415 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b67f33b58f44c66a3745162234cb13d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:04:15 UTC) from {username='oryx', principal='oryx/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:44490
I20260812 19:04:15.392972 3094 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38622:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 19:04:15.393508 3720 tablet_replica.cc:330] T 6b67f33b58f44c66a3745162234cb13d P 0f0aa9470f4b4e3aac661d0089845b9a: stopping tablet replica
W20260812 19:04:15.393417 3094 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:04:15.393854 3718 tablet_replica.cc:330] T 6b67f33b58f44c66a3745162234cb13d P 6183cb8c293e439daea1a4221ba83794: stopping tablet replica
I20260812 19:04:15.393886 3720 raft_consensus.cc:2229] T 6b67f33b58f44c66a3745162234cb13d P 0f0aa9470f4b4e3aac661d0089845b9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:04:15.394246 3718 raft_consensus.cc:2229] T 6b67f33b58f44c66a3745162234cb13d P 6183cb8c293e439daea1a4221ba83794 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:04:15.394255 3720 raft_consensus.cc:2258] T 6b67f33b58f44c66a3745162234cb13d P 0f0aa9470f4b4e3aac661d0089845b9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:04:15.395097 3718 raft_consensus.cc:2258] T 6b67f33b58f44c66a3745162234cb13d P 6183cb8c293e439daea1a4221ba83794 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:04:15.396288 3719 tablet_replica.cc:330] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f: stopping tablet replica
I20260812 19:04:15.396617 3719 raft_consensus.cc:2229] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:04:15.396997 3719 raft_consensus.cc:2258] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:04:15.398100 3718 ts_tablet_manager.cc:1905] T 6b67f33b58f44c66a3745162234cb13d P 6183cb8c293e439daea1a4221ba83794: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:04:15.398921 3250 tablet_service.cc:1467] Processing CreateTablet for tablet 4f08c05636ea48db8665ad6d9d1def76 (DEFAULT_TABLE table=test-table [id=b2cf9f8db0624552aee3a322aec208db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:04:15.399093 3415 tablet_service.cc:1467] Processing CreateTablet for tablet 4f08c05636ea48db8665ad6d9d1def76 (DEFAULT_TABLE table=test-table [id=b2cf9f8db0624552aee3a322aec208db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:04:15.399240 3250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f08c05636ea48db8665ad6d9d1def76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:15.399343 3415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f08c05636ea48db8665ad6d9d1def76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:15.401028 3718 ts_tablet_manager.cc:1918] T 6b67f33b58f44c66a3745162234cb13d P 6183cb8c293e439daea1a4221ba83794: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:04:15.401131 3718 log.cc:1196] T 6b67f33b58f44c66a3745162234cb13d P 6183cb8c293e439daea1a4221ba83794: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-2/wal/wals/6b67f33b58f44c66a3745162234cb13d
I20260812 19:04:15.401903 3692 tablet_bootstrap.cc:492] T 4f08c05636ea48db8665ad6d9d1def76 P 0f0aa9470f4b4e3aac661d0089845b9a: Bootstrap starting.
I20260812 19:04:15.401955 3720 ts_tablet_manager.cc:1905] T 6b67f33b58f44c66a3745162234cb13d P 0f0aa9470f4b4e3aac661d0089845b9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:04:15.402918 3692 tablet_bootstrap.cc:654] T 4f08c05636ea48db8665ad6d9d1def76 P 0f0aa9470f4b4e3aac661d0089845b9a: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:15.403252 3693 tablet_bootstrap.cc:492] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f: Bootstrap starting.
I20260812 19:04:15.404261 3693 tablet_bootstrap.cc:654] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:15.404237 3581 tablet_service.cc:1467] Processing CreateTablet for tablet 4f08c05636ea48db8665ad6d9d1def76 (DEFAULT_TABLE table=test-table [id=b2cf9f8db0624552aee3a322aec208db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:04:15.404515 3581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f08c05636ea48db8665ad6d9d1def76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:15.404521 3720 ts_tablet_manager.cc:1918] T 6b67f33b58f44c66a3745162234cb13d P 0f0aa9470f4b4e3aac661d0089845b9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:04:15.404621 3720 log.cc:1196] T 6b67f33b58f44c66a3745162234cb13d P 0f0aa9470f4b4e3aac661d0089845b9a: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-1/wal/wals/6b67f33b58f44c66a3745162234cb13d
I20260812 19:04:15.404961 3720 ts_tablet_manager.cc:1939] T 6b67f33b58f44c66a3745162234cb13d P 0f0aa9470f4b4e3aac661d0089845b9a: Deleting consensus metadata
I20260812 19:04:15.406318 3718 ts_tablet_manager.cc:1939] T 6b67f33b58f44c66a3745162234cb13d P 6183cb8c293e439daea1a4221ba83794: Deleting consensus metadata
I20260812 19:04:15.407124 3694 tablet_bootstrap.cc:492] T 4f08c05636ea48db8665ad6d9d1def76 P 6183cb8c293e439daea1a4221ba83794: Bootstrap starting.
I20260812 19:04:15.407588 3076 catalog_manager.cc:4841] TS 6183cb8c293e439daea1a4221ba83794 (127.25.130.3:46379): tablet 6b67f33b58f44c66a3745162234cb13d (table test-table [id=169f7a2ecaf44bb0a41d44139d2b1e44]) successfully deleted
I20260812 19:04:15.407605 3077 catalog_manager.cc:4841] TS 0f0aa9470f4b4e3aac661d0089845b9a (127.25.130.2:33077): tablet 6b67f33b58f44c66a3745162234cb13d (table test-table [id=169f7a2ecaf44bb0a41d44139d2b1e44]) successfully deleted
I20260812 19:04:15.408145 3694 tablet_bootstrap.cc:654] T 4f08c05636ea48db8665ad6d9d1def76 P 6183cb8c293e439daea1a4221ba83794: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:15.408445 3719 ts_tablet_manager.cc:1905] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:04:15.409574 3694 tablet_bootstrap.cc:492] T 4f08c05636ea48db8665ad6d9d1def76 P 6183cb8c293e439daea1a4221ba83794: No bootstrap required, opened a new log
I20260812 19:04:15.409680 3694 ts_tablet_manager.cc:1397] T 4f08c05636ea48db8665ad6d9d1def76 P 6183cb8c293e439daea1a4221ba83794: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 19:04:15.410148 3694 raft_consensus.cc:348] T 4f08c05636ea48db8665ad6d9d1def76 P 6183cb8c293e439daea1a4221ba83794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.410271 3694 raft_consensus.cc:374] T 4f08c05636ea48db8665ad6d9d1def76 P 6183cb8c293e439daea1a4221ba83794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:15.410301 3694 raft_consensus.cc:729] T 4f08c05636ea48db8665ad6d9d1def76 P 6183cb8c293e439daea1a4221ba83794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6183cb8c293e439daea1a4221ba83794, State: Initialized, Role: FOLLOWER
I20260812 19:04:15.410460 3694 consensus_queue.cc:260] T 4f08c05636ea48db8665ad6d9d1def76 P 6183cb8c293e439daea1a4221ba83794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.410745 3694 ts_tablet_manager.cc:1428] T 4f08c05636ea48db8665ad6d9d1def76 P 6183cb8c293e439daea1a4221ba83794: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 19:04:15.411054 3719 ts_tablet_manager.cc:1918] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:04:15.411161 3719 log.cc:1196] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-0/wal/wals/6b67f33b58f44c66a3745162234cb13d
I20260812 19:04:15.412039 3719 ts_tablet_manager.cc:1939] T 6b67f33b58f44c66a3745162234cb13d P d074369259f143bba8768daa96c9f43f: Deleting consensus metadata
I20260812 19:04:15.412238 3692 tablet_bootstrap.cc:492] T 4f08c05636ea48db8665ad6d9d1def76 P 0f0aa9470f4b4e3aac661d0089845b9a: No bootstrap required, opened a new log
I20260812 19:04:15.412355 3692 ts_tablet_manager.cc:1397] T 4f08c05636ea48db8665ad6d9d1def76 P 0f0aa9470f4b4e3aac661d0089845b9a: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260812 19:04:15.412879 3692 raft_consensus.cc:348] T 4f08c05636ea48db8665ad6d9d1def76 P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.413012 3692 raft_consensus.cc:374] T 4f08c05636ea48db8665ad6d9d1def76 P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:15.413045 3692 raft_consensus.cc:729] T 4f08c05636ea48db8665ad6d9d1def76 P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f0aa9470f4b4e3aac661d0089845b9a, State: Initialized, Role: FOLLOWER
I20260812 19:04:15.413192 3692 consensus_queue.cc:260] T 4f08c05636ea48db8665ad6d9d1def76 P 0f0aa9470f4b4e3aac661d0089845b9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.413283 3077 catalog_manager.cc:4841] TS d074369259f143bba8768daa96c9f43f (127.25.130.1:37757): tablet 6b67f33b58f44c66a3745162234cb13d (table test-table [id=169f7a2ecaf44bb0a41d44139d2b1e44]) successfully deleted
I20260812 19:04:15.413504 3692 ts_tablet_manager.cc:1428] T 4f08c05636ea48db8665ad6d9d1def76 P 0f0aa9470f4b4e3aac661d0089845b9a: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 19:04:15.414265 3693 tablet_bootstrap.cc:492] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f: No bootstrap required, opened a new log
I20260812 19:04:15.414355 3693 ts_tablet_manager.cc:1397] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.001s
I20260812 19:04:15.414806 3693 raft_consensus.cc:348] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.414906 3693 raft_consensus.cc:374] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:15.414935 3693 raft_consensus.cc:729] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d074369259f143bba8768daa96c9f43f, State: Initialized, Role: FOLLOWER
I20260812 19:04:15.415067 3693 consensus_queue.cc:260] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.415338 3693 ts_tablet_manager.cc:1428] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20260812 19:04:15.446421 3661 tablet.cc:2369] T 4f08c05636ea48db8665ad6d9d1def76 P 6183cb8c293e439daea1a4221ba83794: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 19:04:15.460215 3496 tablet.cc:2369] T 4f08c05636ea48db8665ad6d9d1def76 P 0f0aa9470f4b4e3aac661d0089845b9a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 19:04:15.473031 3329 tablet.cc:2369] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:04:15.529532 3700 raft_consensus.cc:482] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:04:15.529706 3700 raft_consensus.cc:504] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.530182 3700 leader_election.cc:288] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f0aa9470f4b4e3aac661d0089845b9a (127.25.130.2:33077), 6183cb8c293e439daea1a4221ba83794 (127.25.130.3:46379)
I20260812 19:04:15.530645 3606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f08c05636ea48db8665ad6d9d1def76" candidate_uuid: "d074369259f143bba8768daa96c9f43f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6183cb8c293e439daea1a4221ba83794" is_pre_election: true
I20260812 19:04:15.530643 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f08c05636ea48db8665ad6d9d1def76" candidate_uuid: "d074369259f143bba8768daa96c9f43f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" is_pre_election: true
I20260812 19:04:15.530812 3606 raft_consensus.cc:2454] T 4f08c05636ea48db8665ad6d9d1def76 P 6183cb8c293e439daea1a4221ba83794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d074369259f143bba8768daa96c9f43f in term 0.
I20260812 19:04:15.530812 3441 raft_consensus.cc:2454] T 4f08c05636ea48db8665ad6d9d1def76 P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d074369259f143bba8768daa96c9f43f in term 0.
I20260812 19:04:15.531136 3196 leader_election.cc:302] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6183cb8c293e439daea1a4221ba83794, d074369259f143bba8768daa96c9f43f; no voters:
I20260812 19:04:15.531335 3700 raft_consensus.cc:2781] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:04:15.531405 3700 raft_consensus.cc:482] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:04:15.531440 3700 raft_consensus.cc:3037] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:15.532459 3700 raft_consensus.cc:504] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.532815 3700 leader_election.cc:288] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f [CANDIDATE]: Term 1 election: Requested vote from peers 0f0aa9470f4b4e3aac661d0089845b9a (127.25.130.2:33077), 6183cb8c293e439daea1a4221ba83794 (127.25.130.3:46379)
I20260812 19:04:15.533217 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f08c05636ea48db8665ad6d9d1def76" candidate_uuid: "d074369259f143bba8768daa96c9f43f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f0aa9470f4b4e3aac661d0089845b9a"
I20260812 19:04:15.533217 3606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f08c05636ea48db8665ad6d9d1def76" candidate_uuid: "d074369259f143bba8768daa96c9f43f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6183cb8c293e439daea1a4221ba83794"
I20260812 19:04:15.533346 3606 raft_consensus.cc:3037] T 4f08c05636ea48db8665ad6d9d1def76 P 6183cb8c293e439daea1a4221ba83794 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:15.533346 3441 raft_consensus.cc:3037] T 4f08c05636ea48db8665ad6d9d1def76 P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:15.534529 3606 raft_consensus.cc:2454] T 4f08c05636ea48db8665ad6d9d1def76 P 6183cb8c293e439daea1a4221ba83794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d074369259f143bba8768daa96c9f43f in term 1.
I20260812 19:04:15.534610 3441 raft_consensus.cc:2454] T 4f08c05636ea48db8665ad6d9d1def76 P 0f0aa9470f4b4e3aac661d0089845b9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d074369259f143bba8768daa96c9f43f in term 1.
I20260812 19:04:15.534871 3196 leader_election.cc:302] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6183cb8c293e439daea1a4221ba83794, d074369259f143bba8768daa96c9f43f; no voters:
I20260812 19:04:15.535050 3700 raft_consensus.cc:2781] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:04:15.535127 3700 raft_consensus.cc:686] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f [term 1 LEADER]: Becoming Leader. State: Replica: d074369259f143bba8768daa96c9f43f, State: Running, Role: LEADER
I20260812 19:04:15.535262 3700 consensus_queue.cc:237] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.536953 3094 catalog_manager.cc:5495] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f reported cstate change: term changed from 0 to 1, leader changed from <none> to d074369259f143bba8768daa96c9f43f (127.25.130.1). New cstate: current_term: 1 leader_uuid: "d074369259f143bba8768daa96c9f43f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } health_report { overall_health: UNKNOWN } } }
Aug 12 19:04:15 dist-test-slave-49h3 krb5kdc[3039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561454, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.62@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM
I20260812 19:04:15.590104 3152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.575411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38632 (local address 127.25.130.62:44471)
0812 19:04:15.575556 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:15.575561 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:15.575728 (+ 167us) server_negotiation.cc:407] Connection header received
0812 19:04:15.575864 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:15.575866 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:15.575908 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:15.575977 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:15.576670 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.577209 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.578024 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.578244 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.578314 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.586624 (+ 8310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:15.586628 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:15.586630 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:15.586659 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:15.588052 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:15.588648 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:15.588652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:15.588654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:15.588758 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:15.589060 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:15.589063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:15.589064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:15.589299 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:15.589379 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:15.589795 (+ 416us) server_negotiation.cc:299] Negotiation successful
0812 19:04:15.589970 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":60}
I20260812 19:04:15.590108 3731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.575311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:44471 (local address 127.0.0.1:38632)
0812 19:04:15.575638 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:15.575652 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:15.575777 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:15.576024 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:15.576026 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:15.576425 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:15.576538 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.576544 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.577320 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.577323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.577936 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.577946 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.578366 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.578370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.578440 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.584703 (+ 6263us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:15.584721 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:15.586392 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:15.588217 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:15.588230 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:15.588237 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:15.588526 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:15.588877 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:15.588880 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:15.588882 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:15.588978 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:15.589429 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:15.589436 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:15.589613 (+ 177us) client_negotiation.cc:769] Sending connection context
0812 19:04:15.589727 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 19:04:15.589902 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":102,"threads_started":1}
I20260812 19:04:15.594431 3094 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:38632:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 19:04:15.595245 3094 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:04:15.600385 3415 tablet_service.cc:1467] Processing CreateTablet for tablet 491c481b28644ccbbe10a461c2ebcbbb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a2f8e28b2201433f813cb1a586e33567]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 19:04:15.600386 3581 tablet_service.cc:1467] Processing CreateTablet for tablet 491c481b28644ccbbe10a461c2ebcbbb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a2f8e28b2201433f813cb1a586e33567]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 19:04:15.600335 3250 tablet_service.cc:1467] Processing CreateTablet for tablet 491c481b28644ccbbe10a461c2ebcbbb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a2f8e28b2201433f813cb1a586e33567]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 19:04:15.600796 3415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 491c481b28644ccbbe10a461c2ebcbbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:15.600797 3250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 491c481b28644ccbbe10a461c2ebcbbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:15.600797 3581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 491c481b28644ccbbe10a461c2ebcbbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:15.603381 3692 tablet_bootstrap.cc:492] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a: Bootstrap starting.
I20260812 19:04:15.603655 3694 tablet_bootstrap.cc:492] T 491c481b28644ccbbe10a461c2ebcbbb P 6183cb8c293e439daea1a4221ba83794: Bootstrap starting.
I20260812 19:04:15.604377 3692 tablet_bootstrap.cc:654] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:15.604395 3693 tablet_bootstrap.cc:492] T 491c481b28644ccbbe10a461c2ebcbbb P d074369259f143bba8768daa96c9f43f: Bootstrap starting.
I20260812 19:04:15.604627 3694 tablet_bootstrap.cc:654] T 491c481b28644ccbbe10a461c2ebcbbb P 6183cb8c293e439daea1a4221ba83794: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:15.605244 3693 tablet_bootstrap.cc:654] T 491c481b28644ccbbe10a461c2ebcbbb P d074369259f143bba8768daa96c9f43f: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:15.605911 3694 tablet_bootstrap.cc:492] T 491c481b28644ccbbe10a461c2ebcbbb P 6183cb8c293e439daea1a4221ba83794: No bootstrap required, opened a new log
I20260812 19:04:15.605911 3692 tablet_bootstrap.cc:492] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a: No bootstrap required, opened a new log
I20260812 19:04:15.606005 3692 ts_tablet_manager.cc:1397] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 19:04:15.606005 3694 ts_tablet_manager.cc:1397] T 491c481b28644ccbbe10a461c2ebcbbb P 6183cb8c293e439daea1a4221ba83794: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 19:04:15.606393 3693 tablet_bootstrap.cc:492] T 491c481b28644ccbbe10a461c2ebcbbb P d074369259f143bba8768daa96c9f43f: No bootstrap required, opened a new log
I20260812 19:04:15.606482 3693 ts_tablet_manager.cc:1397] T 491c481b28644ccbbe10a461c2ebcbbb P d074369259f143bba8768daa96c9f43f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 19:04:15.606489 3694 raft_consensus.cc:348] T 491c481b28644ccbbe10a461c2ebcbbb P 6183cb8c293e439daea1a4221ba83794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.606489 3692 raft_consensus.cc:348] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.606616 3694 raft_consensus.cc:374] T 491c481b28644ccbbe10a461c2ebcbbb P 6183cb8c293e439daea1a4221ba83794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:15.606616 3692 raft_consensus.cc:374] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:15.606655 3692 raft_consensus.cc:729] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f0aa9470f4b4e3aac661d0089845b9a, State: Initialized, Role: FOLLOWER
I20260812 19:04:15.606655 3694 raft_consensus.cc:729] T 491c481b28644ccbbe10a461c2ebcbbb P 6183cb8c293e439daea1a4221ba83794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6183cb8c293e439daea1a4221ba83794, State: Initialized, Role: FOLLOWER
I20260812 19:04:15.606791 3694 consensus_queue.cc:260] T 491c481b28644ccbbe10a461c2ebcbbb P 6183cb8c293e439daea1a4221ba83794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.606796 3692 consensus_queue.cc:260] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.606894 3693 raft_consensus.cc:348] T 491c481b28644ccbbe10a461c2ebcbbb P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.607012 3693 raft_consensus.cc:374] T 491c481b28644ccbbe10a461c2ebcbbb P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:15.607059 3694 ts_tablet_manager.cc:1428] T 491c481b28644ccbbe10a461c2ebcbbb P 6183cb8c293e439daea1a4221ba83794: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 19:04:15.607061 3693 raft_consensus.cc:729] T 491c481b28644ccbbe10a461c2ebcbbb P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d074369259f143bba8768daa96c9f43f, State: Initialized, Role: FOLLOWER
I20260812 19:04:15.607069 3692 ts_tablet_manager.cc:1428] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 19:04:15.607192 3693 consensus_queue.cc:260] T 491c481b28644ccbbe10a461c2ebcbbb P d074369259f143bba8768daa96c9f43f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.607265 3699 tablet_replica.cc:439] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } } }
I20260812 19:04:15.607285 3701 tablet_replica.cc:439] T 491c481b28644ccbbe10a461c2ebcbbb P 6183cb8c293e439daea1a4221ba83794: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } } }
I20260812 19:04:15.607434 3701 tablet_replica.cc:442] T 491c481b28644ccbbe10a461c2ebcbbb P 6183cb8c293e439daea1a4221ba83794: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 19:04:15.607434 3699 tablet_replica.cc:442] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 19:04:15.607434 3693 ts_tablet_manager.cc:1428] T 491c481b28644ccbbe10a461c2ebcbbb P d074369259f143bba8768daa96c9f43f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 19:04:15.607641 3700 tablet_replica.cc:439] T 491c481b28644ccbbe10a461c2ebcbbb P d074369259f143bba8768daa96c9f43f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } } }
I20260812 19:04:15.607752 3700 tablet_replica.cc:442] T 491c481b28644ccbbe10a461c2ebcbbb P d074369259f143bba8768daa96c9f43f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 19:04:15.649399 3699 raft_consensus.cc:482] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:04:15.649600 3699 raft_consensus.cc:504] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.650749 3699 leader_election.cc:288] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d074369259f143bba8768daa96c9f43f (127.25.130.1:37757), 6183cb8c293e439daea1a4221ba83794 (127.25.130.3:46379)
I20260812 19:04:15.654708 3690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.651103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:33917 (local address 127.25.130.3:46379)
0812 19:04:15.651238 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:15.651243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:15.651491 (+ 248us) server_negotiation.cc:407] Connection header received
0812 19:04:15.651632 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:15.651635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:15.651681 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:15.651752 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:15.652040 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.652541 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.653758 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.654155 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.654252 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.654307 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:15.654488 (+ 181us) server_negotiation.cc:299] Negotiation successful
0812 19:04:15.654565 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 19:04:15.654717 3691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.651072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:46669 (local address 127.25.130.1:37757)
0812 19:04:15.651207 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:15.651211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:15.651429 (+ 218us) server_negotiation.cc:407] Connection header received
0812 19:04:15.651588 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:15.651592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:15.651649 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:15.651710 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:15.652057 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.652555 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.653788 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.654209 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.654327 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.654383 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:15.654526 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 19:04:15.654610 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 19:04:15.654731 3735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.650997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:46379 (local address 127.25.130.2:33917)
0812 19:04:15.651411 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:15.651427 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:15.651541 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:15.651771 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:15.651774 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:15.651815 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:15.651935 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.651939 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.652645 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.652655 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.653642 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.653652 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.654270 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.654274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.654368 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.654392 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 19:04:15.654462 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 19:04:15.654549 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":54,"spinlock_wait_cycles":1536,"thread_start_us":79,"threads_started":1}
I20260812 19:04:15.654731 3736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.650962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:37757 (local address 127.25.130.2:46669)
0812 19:04:15.651327 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:15.651339 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:15.651439 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:15.651743 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:15.651746 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:15.651762 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:15.651911 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.651916 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.652662 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.652665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.653660 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.653670 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.654327 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.654331 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.654411 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.654428 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 19:04:15.654499 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 19:04:15.654565 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":70,"spinlock_wait_cycles":18176,"thread_start_us":131,"threads_started":1}
I20260812 19:04:15.655344 3606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "491c481b28644ccbbe10a461c2ebcbbb" candidate_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6183cb8c293e439daea1a4221ba83794" is_pre_election: true
I20260812 19:04:15.655515 3606 raft_consensus.cc:2454] T 491c481b28644ccbbe10a461c2ebcbbb P 6183cb8c293e439daea1a4221ba83794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f0aa9470f4b4e3aac661d0089845b9a in term 0.
I20260812 19:04:15.655866 3276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "491c481b28644ccbbe10a461c2ebcbbb" candidate_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d074369259f143bba8768daa96c9f43f" is_pre_election: true
I20260812 19:04:15.656150 3276 raft_consensus.cc:2454] T 491c481b28644ccbbe10a461c2ebcbbb P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f0aa9470f4b4e3aac661d0089845b9a in term 0.
I20260812 19:04:15.656581 3364 leader_election.cc:302] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f0aa9470f4b4e3aac661d0089845b9a, d074369259f143bba8768daa96c9f43f; no voters:
I20260812 19:04:15.656865 3699 raft_consensus.cc:2781] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:04:15.656929 3699 raft_consensus.cc:482] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:04:15.656967 3699 raft_consensus.cc:3037] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:15.658039 3699 raft_consensus.cc:504] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.658373 3699 leader_election.cc:288] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a [CANDIDATE]: Term 1 election: Requested vote from peers d074369259f143bba8768daa96c9f43f (127.25.130.1:37757), 6183cb8c293e439daea1a4221ba83794 (127.25.130.3:46379)
I20260812 19:04:15.658711 3606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "491c481b28644ccbbe10a461c2ebcbbb" candidate_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6183cb8c293e439daea1a4221ba83794"
I20260812 19:04:15.658739 3276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "491c481b28644ccbbe10a461c2ebcbbb" candidate_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d074369259f143bba8768daa96c9f43f"
I20260812 19:04:15.658823 3606 raft_consensus.cc:3037] T 491c481b28644ccbbe10a461c2ebcbbb P 6183cb8c293e439daea1a4221ba83794 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:15.658895 3276 raft_consensus.cc:3037] T 491c481b28644ccbbe10a461c2ebcbbb P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:15.659705 3606 raft_consensus.cc:2454] T 491c481b28644ccbbe10a461c2ebcbbb P 6183cb8c293e439daea1a4221ba83794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f0aa9470f4b4e3aac661d0089845b9a in term 1.
I20260812 19:04:15.659966 3276 raft_consensus.cc:2454] T 491c481b28644ccbbe10a461c2ebcbbb P d074369259f143bba8768daa96c9f43f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f0aa9470f4b4e3aac661d0089845b9a in term 1.
I20260812 19:04:15.660005 3363 leader_election.cc:302] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f0aa9470f4b4e3aac661d0089845b9a, 6183cb8c293e439daea1a4221ba83794; no voters:
I20260812 19:04:15.660161 3699 raft_consensus.cc:2781] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:04:15.660332 3699 raft_consensus.cc:686] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a [term 1 LEADER]: Becoming Leader. State: Replica: 0f0aa9470f4b4e3aac661d0089845b9a, State: Running, Role: LEADER
I20260812 19:04:15.660575 3699 consensus_queue.cc:237] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.661825 3737 tablet_replica.cc:439] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a: TxnStatusTablet state changed. Reason: New leader 0f0aa9470f4b4e3aac661d0089845b9a. Latest consensus state: current_term: 1 leader_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } } }
I20260812 19:04:15.661942 3737 tablet_replica.cc:442] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a: This TxnStatusTablet replica's current role is: LEADER
I20260812 19:04:15.662226 3742 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 19:04:15.662384 3742 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 19:04:15.663033 3091 catalog_manager.cc:5495] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f0aa9470f4b4e3aac661d0089845b9a (127.25.130.2). New cstate: current_term: 1 leader_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } health_report { overall_health: UNKNOWN } } }
I20260812 19:04:15.678053 3731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.674391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:33077 (local address 127.0.0.1:44500)
0812 19:04:15.674562 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:15.674579 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:15.674694 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:15.675007 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:15.675010 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:15.675019 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 19:04:15.675129 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.675134 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.675840 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.675843 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.676449 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.676458 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.676890 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.676894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.676979 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.677029 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 19:04:15.677741 (+ 712us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 19:04:15.677759 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 19:04:15.677825 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 19:04:15.677922 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 19:04:15.678053 3687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.674501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44500 (local address 127.25.130.2:33077)
0812 19:04:15.674640 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:15.674644 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:15.674658 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:04:15.674836 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:15.674839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:15.674900 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:15.674983 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:04:15.675228 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.675735 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.676568 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.676791 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.676877 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.677170 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:04:15.677649 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:04:15.677754 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:15.677845 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 19:04:15.677900 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 19:04:15.681061 3606 raft_consensus.cc:1261] T 491c481b28644ccbbe10a461c2ebcbbb P 6183cb8c293e439daea1a4221ba83794 [term 1 FOLLOWER]: Refusing update from remote peer 0f0aa9470f4b4e3aac661d0089845b9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:04:15.681313 3276 raft_consensus.cc:1261] T 491c481b28644ccbbe10a461c2ebcbbb P d074369259f143bba8768daa96c9f43f [term 1 FOLLOWER]: Refusing update from remote peer 0f0aa9470f4b4e3aac661d0089845b9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:04:15.681578 3699 consensus_queue.cc:1035] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:04:15.681931 3737 consensus_queue.cc:1035] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:04:15.683944 3700 tablet_replica.cc:439] T 491c481b28644ccbbe10a461c2ebcbbb P d074369259f143bba8768daa96c9f43f: TxnStatusTablet state changed. Reason: New leader 0f0aa9470f4b4e3aac661d0089845b9a. Latest consensus state: current_term: 1 leader_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } } }
I20260812 19:04:15.684075 3700 tablet_replica.cc:442] T 491c481b28644ccbbe10a461c2ebcbbb P d074369259f143bba8768daa96c9f43f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 19:04:15.685210 3701 tablet_replica.cc:439] T 491c481b28644ccbbe10a461c2ebcbbb P 6183cb8c293e439daea1a4221ba83794: TxnStatusTablet state changed. Reason: New leader 0f0aa9470f4b4e3aac661d0089845b9a. Latest consensus state: current_term: 1 leader_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } } }
I20260812 19:04:15.685341 3701 tablet_replica.cc:442] T 491c481b28644ccbbe10a461c2ebcbbb P 6183cb8c293e439daea1a4221ba83794: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 19:04:15.686805 3699 tablet_replica.cc:439] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } } }
I20260812 19:04:15.686928 3699 tablet_replica.cc:442] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a: This TxnStatusTablet replica's current role is: LEADER
I20260812 19:04:15.687606 3709 mvcc.cc:205] Tried to move back new op lower bound from 7317755722464681984 to 7317755722388148224. Current Snapshot: MvccSnapshot[applied={T|T < 7317755722464681984 or (T in {7317755722464681984})}]
I20260812 19:04:15.687748 3699 tablet_replica.cc:439] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } } }
I20260812 19:04:15.687867 3699 tablet_replica.cc:442] T 491c481b28644ccbbe10a461c2ebcbbb P 0f0aa9470f4b4e3aac661d0089845b9a: This TxnStatusTablet replica's current role is: LEADER
I20260812 19:04:15.688095 3710 mvcc.cc:205] Tried to move back new op lower bound from 7317755722464681984 to 7317755722388148224. Current Snapshot: MvccSnapshot[applied={T|T < 7317755722464681984}]
I20260812 19:04:15.688218 3701 tablet_replica.cc:439] T 491c481b28644ccbbe10a461c2ebcbbb P 6183cb8c293e439daea1a4221ba83794: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } } }
I20260812 19:04:15.688323 3701 tablet_replica.cc:442] T 491c481b28644ccbbe10a461c2ebcbbb P 6183cb8c293e439daea1a4221ba83794: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 19:04:15.689424 3707 mvcc.cc:205] Tried to move back new op lower bound from 7317755722464681984 to 7317755722388148224. Current Snapshot: MvccSnapshot[applied={T|T < 7317755722464681984}]
I20260812 19:04:15.689652 3700 tablet_replica.cc:439] T 491c481b28644ccbbe10a461c2ebcbbb P d074369259f143bba8768daa96c9f43f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } } }
I20260812 19:04:15.689754 3700 tablet_replica.cc:442] T 491c481b28644ccbbe10a461c2ebcbbb P d074369259f143bba8768daa96c9f43f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 19:04:15.704300 3687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.701031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:46733 (local address 127.25.130.2:33077)
0812 19:04:15.701292 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:15.701296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:15.701314 (+ 18us) server_negotiation.cc:407] Connection header received
0812 19:04:15.701506 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:15.701509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:15.701559 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:15.701621 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:04:15.701876 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.702386 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.703250 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.703472 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.703559 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.703857 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:04:15.703936 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:04:15.703994 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:15.704089 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 19:04:15.704148 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157}
I20260812 19:04:15.704305 3749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.700917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:33077 (local address 127.25.130.1:46733)
0812 19:04:15.701249 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:15.701266 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:15.701374 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:15.701650 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:15.701653 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:15.701661 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 19:04:15.701775 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.701782 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.702496 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.702499 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.703133 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.703143 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.703575 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.703579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.703676 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.703727 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 19:04:15.704011 (+ 284us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 19:04:15.704018 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 19:04:15.704078 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 19:04:15.704174 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":105,"threads_started":1}
I20260812 19:04:15.711913 3606 raft_consensus.cc:1261] T 4f08c05636ea48db8665ad6d9d1def76 P 6183cb8c293e439daea1a4221ba83794 [term 1 FOLLOWER]: Refusing update from remote peer d074369259f143bba8768daa96c9f43f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:04:15.712076 3441 raft_consensus.cc:1261] T 4f08c05636ea48db8665ad6d9d1def76 P 0f0aa9470f4b4e3aac661d0089845b9a [term 1 FOLLOWER]: Refusing update from remote peer d074369259f143bba8768daa96c9f43f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:04:15.712540 3704 consensus_queue.cc:1035] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:04:15.712733 3700 consensus_queue.cc:1035] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:04:15.740137 3691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.735487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:44095 (local address 127.25.130.1:37757)
0812 19:04:15.735596 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:15.735600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:15.735918 (+ 318us) server_negotiation.cc:407] Connection header received
0812 19:04:15.736067 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:15.736070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:15.736123 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:15.736196 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:04:15.736666 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.737504 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.738787 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.739167 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.739269 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.739627 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:04:15.739724 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:04:15.739767 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:15.739919 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 19:04:15.739989 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 19:04:15.740623 3757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.735414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:37757 (local address 127.25.130.2:44095)
0812 19:04:15.735834 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:15.735849 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:15.735963 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:15.736366 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:15.736370 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:15.736378 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 19:04:15.736532 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.736539 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.737654 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.737657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.738659 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.738671 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.739281 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.739285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.739384 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.739448 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 19:04:15.739826 (+ 378us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 19:04:15.739832 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 19:04:15.740363 (+ 531us) client_negotiation.cc:240] Negotiation successful
0812 19:04:15.740455 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":128,"threads_started":1}
I20260812 19:04:15.773532 3091 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:04:15.773698 3091 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:04:15.776126 3091 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b1beb356039e4f5185921f11c188d34b: Sending DeleteTablet for 3 replicas of tablet 4f08c05636ea48db8665ad6d9d1def76
I20260812 19:04:15.776895 3250 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f08c05636ea48db8665ad6d9d1def76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:04:15 UTC) from {username='oryx', principal='oryx/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:60240
I20260812 19:04:15.776895 3415 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f08c05636ea48db8665ad6d9d1def76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:04:15 UTC) from {username='oryx', principal='oryx/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:44490
I20260812 19:04:15.776948 3581 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f08c05636ea48db8665ad6d9d1def76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:04:15 UTC) from {username='oryx', principal='oryx/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:41036
I20260812 19:04:15.777112 3719 tablet_replica.cc:330] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f: stopping tablet replica
I20260812 19:04:15.777114 3720 tablet_replica.cc:330] T 4f08c05636ea48db8665ad6d9d1def76 P 0f0aa9470f4b4e3aac661d0089845b9a: stopping tablet replica
I20260812 19:04:15.777240 3720 raft_consensus.cc:2229] T 4f08c05636ea48db8665ad6d9d1def76 P 0f0aa9470f4b4e3aac661d0089845b9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:04:15.777256 3719 raft_consensus.cc:2229] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:04:15.777385 3720 raft_consensus.cc:2258] T 4f08c05636ea48db8665ad6d9d1def76 P 0f0aa9470f4b4e3aac661d0089845b9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:04:15.777513 3719 raft_consensus.cc:2258] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:04:15.777598 3718 tablet_replica.cc:330] T 4f08c05636ea48db8665ad6d9d1def76 P 6183cb8c293e439daea1a4221ba83794: stopping tablet replica
I20260812 19:04:15.777732 3718 raft_consensus.cc:2229] T 4f08c05636ea48db8665ad6d9d1def76 P 6183cb8c293e439daea1a4221ba83794 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:04:15.777854 3718 raft_consensus.cc:2258] T 4f08c05636ea48db8665ad6d9d1def76 P 6183cb8c293e439daea1a4221ba83794 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:04:15.778625 3718 ts_tablet_manager.cc:1905] T 4f08c05636ea48db8665ad6d9d1def76 P 6183cb8c293e439daea1a4221ba83794: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:04:15.778677 3719 ts_tablet_manager.cc:1905] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:04:15.781380 3719 ts_tablet_manager.cc:1918] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 19:04:15.781478 3719 log.cc:1196] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-0/wal/wals/4f08c05636ea48db8665ad6d9d1def76
I20260812 19:04:15.781638 3718 ts_tablet_manager.cc:1918] T 4f08c05636ea48db8665ad6d9d1def76 P 6183cb8c293e439daea1a4221ba83794: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 19:04:15.781713 3718 log.cc:1196] T 4f08c05636ea48db8665ad6d9d1def76 P 6183cb8c293e439daea1a4221ba83794: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-2/wal/wals/4f08c05636ea48db8665ad6d9d1def76
I20260812 19:04:15.781729 3719 ts_tablet_manager.cc:1939] T 4f08c05636ea48db8665ad6d9d1def76 P d074369259f143bba8768daa96c9f43f: Deleting consensus metadata
I20260812 19:04:15.781950 3718 ts_tablet_manager.cc:1939] T 4f08c05636ea48db8665ad6d9d1def76 P 6183cb8c293e439daea1a4221ba83794: Deleting consensus metadata
I20260812 19:04:15.782472 3720 ts_tablet_manager.cc:1905] T 4f08c05636ea48db8665ad6d9d1def76 P 0f0aa9470f4b4e3aac661d0089845b9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:04:15.783003 3076 catalog_manager.cc:4841] TS 6183cb8c293e439daea1a4221ba83794 (127.25.130.3:46379): tablet 4f08c05636ea48db8665ad6d9d1def76 (table test-table [id=b2cf9f8db0624552aee3a322aec208db]) successfully deleted
I20260812 19:04:15.789417 3720 ts_tablet_manager.cc:1918] T 4f08c05636ea48db8665ad6d9d1def76 P 0f0aa9470f4b4e3aac661d0089845b9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 19:04:15.789547 3720 log.cc:1196] T 4f08c05636ea48db8665ad6d9d1def76 P 0f0aa9470f4b4e3aac661d0089845b9a: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-1/wal/wals/4f08c05636ea48db8665ad6d9d1def76
I20260812 19:04:15.789826 3720 ts_tablet_manager.cc:1939] T 4f08c05636ea48db8665ad6d9d1def76 P 0f0aa9470f4b4e3aac661d0089845b9a: Deleting consensus metadata
I20260812 19:04:15.790359 3077 catalog_manager.cc:4841] TS d074369259f143bba8768daa96c9f43f (127.25.130.1:37757): tablet 4f08c05636ea48db8665ad6d9d1def76 (table test-table [id=b2cf9f8db0624552aee3a322aec208db]) successfully deleted
I20260812 19:04:15.790567 3077 catalog_manager.cc:4841] TS 0f0aa9470f4b4e3aac661d0089845b9a (127.25.130.2:33077): tablet 4f08c05636ea48db8665ad6d9d1def76 (table test-table [id=b2cf9f8db0624552aee3a322aec208db]) successfully deleted
Aug 12 19:04:15 dist-test-slave-49h3 krb5kdc[3039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 19:04:15 dist-test-slave-49h3 krb5kdc[3039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:15 dist-test-slave-49h3 krb5kdc[3039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
W20260812 19:04:15.818722 3152 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:15.811978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47206 (local address 127.25.130.62:44471)
0812 19:04:15.812114 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:15.812120 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:15.812377 (+ 257us) server_negotiation.cc:407] Connection header received
0812 19:04:15.812533 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:15.812536 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:15.812585 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:15.812676 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:15.813525 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.814116 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.815001 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.815282 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.815372 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.818599 (+ 3227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47206: BlockingRecv error: recv got EOF from 127.0.0.1:47206 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 12 19:04:15 dist-test-slave-49h3 krb5kdc[3039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM
I20260812 19:04:15.829937 3152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.821013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47220 (local address 127.25.130.62:44471)
0812 19:04:15.821162 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:15.821166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:15.821181 (+ 15us) server_negotiation.cc:407] Connection header received
0812 19:04:15.821296 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:15.821299 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:15.821351 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:15.821416 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:15.822163 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.822884 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.824039 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.824281 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.824370 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.826917 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:15.826921 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:15.826924 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:15.826955 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:15.828277 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:15.828677 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:15.828680 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:15.828683 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:15.828767 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:15.829015 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:15.829018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:15.829020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:15.829225 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:15.829345 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:15.829654 (+ 309us) server_negotiation.cc:299] Negotiation successful
0812 19:04:15.829777 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 19:04:15.832715 3091 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:04:15.833196 3091 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:04:15.838086 3250 tablet_service.cc:1467] Processing CreateTablet for tablet 7b317b74eabe411cbf0ecaaeef0c3b07 (DEFAULT_TABLE table=test-table [id=3b5e4376a17d44a18a730ceab3b2772e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:04:15.838126 3415 tablet_service.cc:1467] Processing CreateTablet for tablet 7b317b74eabe411cbf0ecaaeef0c3b07 (DEFAULT_TABLE table=test-table [id=3b5e4376a17d44a18a730ceab3b2772e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:04:15.838274 3581 tablet_service.cc:1467] Processing CreateTablet for tablet 7b317b74eabe411cbf0ecaaeef0c3b07 (DEFAULT_TABLE table=test-table [id=3b5e4376a17d44a18a730ceab3b2772e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:04:15.838449 3250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b317b74eabe411cbf0ecaaeef0c3b07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:15.838554 3581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b317b74eabe411cbf0ecaaeef0c3b07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:15.838524 3415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b317b74eabe411cbf0ecaaeef0c3b07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:15.840916 3693 tablet_bootstrap.cc:492] T 7b317b74eabe411cbf0ecaaeef0c3b07 P d074369259f143bba8768daa96c9f43f: Bootstrap starting.
I20260812 19:04:15.841006 3694 tablet_bootstrap.cc:492] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 6183cb8c293e439daea1a4221ba83794: Bootstrap starting.
I20260812 19:04:15.841892 3692 tablet_bootstrap.cc:492] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a: Bootstrap starting.
I20260812 19:04:15.842006 3694 tablet_bootstrap.cc:654] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 6183cb8c293e439daea1a4221ba83794: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:15.842749 3692 tablet_bootstrap.cc:654] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:15.843672 3694 tablet_bootstrap.cc:492] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 6183cb8c293e439daea1a4221ba83794: No bootstrap required, opened a new log
I20260812 19:04:15.843765 3694 ts_tablet_manager.cc:1397] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 6183cb8c293e439daea1a4221ba83794: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 19:04:15.843828 3693 tablet_bootstrap.cc:654] T 7b317b74eabe411cbf0ecaaeef0c3b07 P d074369259f143bba8768daa96c9f43f: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:15.844416 3694 raft_consensus.cc:348] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 6183cb8c293e439daea1a4221ba83794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.844534 3694 raft_consensus.cc:374] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 6183cb8c293e439daea1a4221ba83794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:15.844568 3694 raft_consensus.cc:729] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 6183cb8c293e439daea1a4221ba83794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6183cb8c293e439daea1a4221ba83794, State: Initialized, Role: FOLLOWER
I20260812 19:04:15.844704 3694 consensus_queue.cc:260] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 6183cb8c293e439daea1a4221ba83794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.845202 3692 tablet_bootstrap.cc:492] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a: No bootstrap required, opened a new log
I20260812 19:04:15.845291 3692 ts_tablet_manager.cc:1397] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 19:04:15.845288 3694 ts_tablet_manager.cc:1428] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 6183cb8c293e439daea1a4221ba83794: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 19:04:15.845893 3693 tablet_bootstrap.cc:492] T 7b317b74eabe411cbf0ecaaeef0c3b07 P d074369259f143bba8768daa96c9f43f: No bootstrap required, opened a new log
I20260812 19:04:15.845984 3693 ts_tablet_manager.cc:1397] T 7b317b74eabe411cbf0ecaaeef0c3b07 P d074369259f143bba8768daa96c9f43f: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260812 19:04:15.846391 3693 raft_consensus.cc:348] T 7b317b74eabe411cbf0ecaaeef0c3b07 P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.846500 3693 raft_consensus.cc:374] T 7b317b74eabe411cbf0ecaaeef0c3b07 P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:15.846531 3693 raft_consensus.cc:729] T 7b317b74eabe411cbf0ecaaeef0c3b07 P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d074369259f143bba8768daa96c9f43f, State: Initialized, Role: FOLLOWER
I20260812 19:04:15.846652 3693 consensus_queue.cc:260] T 7b317b74eabe411cbf0ecaaeef0c3b07 P d074369259f143bba8768daa96c9f43f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.846935 3693 ts_tablet_manager.cc:1428] T 7b317b74eabe411cbf0ecaaeef0c3b07 P d074369259f143bba8768daa96c9f43f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 19:04:15.847808 3692 raft_consensus.cc:348] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.847935 3692 raft_consensus.cc:374] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:15.847963 3692 raft_consensus.cc:729] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f0aa9470f4b4e3aac661d0089845b9a, State: Initialized, Role: FOLLOWER
I20260812 19:04:15.848083 3692 consensus_queue.cc:260] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.848362 3692 ts_tablet_manager.cc:1428] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260812 19:04:15.924903 3737 raft_consensus.cc:482] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:04:15.925033 3737 raft_consensus.cc:504] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.925235 3701 raft_consensus.cc:482] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 6183cb8c293e439daea1a4221ba83794 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:04:15.925359 3701 raft_consensus.cc:504] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 6183cb8c293e439daea1a4221ba83794 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.925524 3737 leader_election.cc:288] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d074369259f143bba8768daa96c9f43f (127.25.130.1:37757), 6183cb8c293e439daea1a4221ba83794 (127.25.130.3:46379)
I20260812 19:04:15.925949 3606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b317b74eabe411cbf0ecaaeef0c3b07" candidate_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6183cb8c293e439daea1a4221ba83794" is_pre_election: true
I20260812 19:04:15.925951 3276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b317b74eabe411cbf0ecaaeef0c3b07" candidate_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d074369259f143bba8768daa96c9f43f" is_pre_election: true
I20260812 19:04:15.926097 3606 raft_consensus.cc:2454] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 6183cb8c293e439daea1a4221ba83794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f0aa9470f4b4e3aac661d0089845b9a in term 0.
I20260812 19:04:15.926098 3276 raft_consensus.cc:2454] T 7b317b74eabe411cbf0ecaaeef0c3b07 P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f0aa9470f4b4e3aac661d0089845b9a in term 0.
I20260812 19:04:15.926407 3364 leader_election.cc:302] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0f0aa9470f4b4e3aac661d0089845b9a, 6183cb8c293e439daea1a4221ba83794, d074369259f143bba8768daa96c9f43f; no voters:
I20260812 19:04:15.926582 3737 raft_consensus.cc:2781] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:04:15.926642 3737 raft_consensus.cc:482] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:04:15.926656 3701 leader_election.cc:288] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 6183cb8c293e439daea1a4221ba83794 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d074369259f143bba8768daa96c9f43f (127.25.130.1:37757), 0f0aa9470f4b4e3aac661d0089845b9a (127.25.130.2:33077)
I20260812 19:04:15.926684 3737 raft_consensus.cc:3037] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:15.928125 3737 raft_consensus.cc:504] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.928440 3737 leader_election.cc:288] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a [CANDIDATE]: Term 1 election: Requested vote from peers d074369259f143bba8768daa96c9f43f (127.25.130.1:37757), 6183cb8c293e439daea1a4221ba83794 (127.25.130.3:46379)
I20260812 19:04:15.928795 3606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b317b74eabe411cbf0ecaaeef0c3b07" candidate_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6183cb8c293e439daea1a4221ba83794"
I20260812 19:04:15.928787 3276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b317b74eabe411cbf0ecaaeef0c3b07" candidate_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d074369259f143bba8768daa96c9f43f"
I20260812 19:04:15.928910 3606 raft_consensus.cc:3037] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 6183cb8c293e439daea1a4221ba83794 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:15.928912 3276 raft_consensus.cc:3037] T 7b317b74eabe411cbf0ecaaeef0c3b07 P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:15.930042 3606 raft_consensus.cc:2454] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 6183cb8c293e439daea1a4221ba83794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f0aa9470f4b4e3aac661d0089845b9a in term 1.
I20260812 19:04:15.930111 3276 raft_consensus.cc:2454] T 7b317b74eabe411cbf0ecaaeef0c3b07 P d074369259f143bba8768daa96c9f43f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f0aa9470f4b4e3aac661d0089845b9a in term 1.
I20260812 19:04:15.930385 3363 leader_election.cc:302] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f0aa9470f4b4e3aac661d0089845b9a, 6183cb8c293e439daea1a4221ba83794; no voters:
I20260812 19:04:15.930650 3737 raft_consensus.cc:2781] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:04:15.930727 3737 raft_consensus.cc:686] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a [term 1 LEADER]: Becoming Leader. State: Replica: 0f0aa9470f4b4e3aac661d0089845b9a, State: Running, Role: LEADER
I20260812 19:04:15.930855 3737 consensus_queue.cc:237] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.931233 3691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.926916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:53623 (local address 127.25.130.1:37757)
0812 19:04:15.927056 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:15.927061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:15.927254 (+ 193us) server_negotiation.cc:407] Connection header received
0812 19:04:15.927460 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:15.927464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:15.927520 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:15.927604 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:15.927906 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.928523 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.929986 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.930566 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.930648 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.930701 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:15.931011 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 19:04:15.931096 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 19:04:15.931660 3775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.926828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:37757 (local address 127.25.130.3:53623)
0812 19:04:15.927165 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:15.927181 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:15.927326 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:15.927626 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:15.927629 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:15.927667 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:15.927788 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.927795 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.928643 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.928647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.929846 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.929857 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.930799 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.930802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.930877 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.930900 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 19:04:15.931407 (+ 507us) client_negotiation.cc:240] Negotiation successful
0812 19:04:15.931509 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"spinlock_wait_cycles":15872,"thread_start_us":85,"threads_started":1}
I20260812 19:04:15.932039 3276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b317b74eabe411cbf0ecaaeef0c3b07" candidate_uuid: "6183cb8c293e439daea1a4221ba83794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d074369259f143bba8768daa96c9f43f" is_pre_election: true
I20260812 19:04:15.932228 3276 raft_consensus.cc:2379] T 7b317b74eabe411cbf0ecaaeef0c3b07 P d074369259f143bba8768daa96c9f43f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6183cb8c293e439daea1a4221ba83794 in current term 1: Already voted for candidate 0f0aa9470f4b4e3aac661d0089845b9a in this term.
I20260812 19:04:15.932257 3687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.927272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:41449 (local address 127.25.130.2:33077)
0812 19:04:15.927375 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:15.927379 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:15.927601 (+ 222us) server_negotiation.cc:407] Connection header received
0812 19:04:15.927769 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:15.927772 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:15.927814 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:15.927879 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:15.928859 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.929556 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.930910 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.931454 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.931559 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.931606 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:15.932067 (+ 461us) server_negotiation.cc:299] Negotiation successful
0812 19:04:15.932143 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 19:04:15.932610 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.927196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:33077 (local address 127.25.130.3:41449)
0812 19:04:15.927530 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:15.927542 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:15.927680 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:15.928585 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:15.928587 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:15.928607 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:15.928731 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.928738 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.929704 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.929707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.930786 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:15.930796 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.931705 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.931708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:15.931927 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.931951 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 19:04:15.932410 (+ 459us) client_negotiation.cc:240] Negotiation successful
0812 19:04:15.932484 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":1,"spinlock_wait_cycles":896,"thread_start_us":73,"threads_started":1}
I20260812 19:04:15.932819 3091 catalog_manager.cc:5495] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f0aa9470f4b4e3aac661d0089845b9a (127.25.130.2). New cstate: current_term: 1 leader_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } health_report { overall_health: UNKNOWN } } }
I20260812 19:04:15.933012 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b317b74eabe411cbf0ecaaeef0c3b07" candidate_uuid: "6183cb8c293e439daea1a4221ba83794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" is_pre_election: true
I20260812 19:04:15.933362 3528 leader_election.cc:302] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 6183cb8c293e439daea1a4221ba83794 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6183cb8c293e439daea1a4221ba83794; no voters: 0f0aa9470f4b4e3aac661d0089845b9a, d074369259f143bba8768daa96c9f43f
I20260812 19:04:15.933682 3701 raft_consensus.cc:2726] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 6183cb8c293e439daea1a4221ba83794 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:04:15.950474 3687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:15.947379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38742 (local address 127.25.130.2:33077)
0812 19:04:15.947506 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:15.947511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:15.947525 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:04:15.947664 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:15.947667 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:15.947717 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:15.947782 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:04:15.948055 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.948575 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.949419 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:15.949688 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:15.949775 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:15.950035 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:04:15.950109 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:04:15.950175 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:15.950269 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 19:04:15.950330 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 19:04:15.952872 3276 raft_consensus.cc:1261] T 7b317b74eabe411cbf0ecaaeef0c3b07 P d074369259f143bba8768daa96c9f43f [term 1 FOLLOWER]: Refusing update from remote peer 0f0aa9470f4b4e3aac661d0089845b9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:04:15.952872 3606 raft_consensus.cc:1261] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 6183cb8c293e439daea1a4221ba83794 [term 1 FOLLOWER]: Refusing update from remote peer 0f0aa9470f4b4e3aac661d0089845b9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:04:15.953316 3737 consensus_queue.cc:1035] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:04:15.953547 3751 consensus_queue.cc:1035] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:04:15.963500 3094 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:04:15.963665 3094 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:04:15.965893 3094 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b1beb356039e4f5185921f11c188d34b: Sending DeleteTablet for 3 replicas of tablet 7b317b74eabe411cbf0ecaaeef0c3b07
I20260812 19:04:15.966531 3415 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b317b74eabe411cbf0ecaaeef0c3b07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:04:15 UTC) from {username='oryx', principal='oryx/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:44490
I20260812 19:04:15.966719 3720 tablet_replica.cc:330] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a: stopping tablet replica
I20260812 19:04:15.966712 3250 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b317b74eabe411cbf0ecaaeef0c3b07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:04:15 UTC) from {username='oryx', principal='oryx/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:60240
I20260812 19:04:15.966719 3581 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b317b74eabe411cbf0ecaaeef0c3b07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:04:15 UTC) from {username='oryx', principal='oryx/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:41036
I20260812 19:04:15.966836 3720 raft_consensus.cc:2229] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:04:15.966902 3719 tablet_replica.cc:330] T 7b317b74eabe411cbf0ecaaeef0c3b07 P d074369259f143bba8768daa96c9f43f: stopping tablet replica
I20260812 19:04:15.967027 3719 raft_consensus.cc:2229] T 7b317b74eabe411cbf0ecaaeef0c3b07 P d074369259f143bba8768daa96c9f43f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:04:15.967180 3719 raft_consensus.cc:2258] T 7b317b74eabe411cbf0ecaaeef0c3b07 P d074369259f143bba8768daa96c9f43f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:04:15.967183 3720 raft_consensus.cc:2258] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:04:15.967367 3718 tablet_replica.cc:330] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 6183cb8c293e439daea1a4221ba83794: stopping tablet replica
I20260812 19:04:15.967480 3718 raft_consensus.cc:2229] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 6183cb8c293e439daea1a4221ba83794 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:04:15.967599 3718 raft_consensus.cc:2258] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 6183cb8c293e439daea1a4221ba83794 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:04:15.968375 3718 ts_tablet_manager.cc:1905] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 6183cb8c293e439daea1a4221ba83794: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:04:15.968407 3720 ts_tablet_manager.cc:1905] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:04:15.970465 3720 ts_tablet_manager.cc:1918] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:04:15.970544 3720 log.cc:1196] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-1/wal/wals/7b317b74eabe411cbf0ecaaeef0c3b07
I20260812 19:04:15.970762 3720 ts_tablet_manager.cc:1939] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 0f0aa9470f4b4e3aac661d0089845b9a: Deleting consensus metadata
I20260812 19:04:15.970901 3718 ts_tablet_manager.cc:1918] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 6183cb8c293e439daea1a4221ba83794: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:04:15.970991 3718 log.cc:1196] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 6183cb8c293e439daea1a4221ba83794: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-2/wal/wals/7b317b74eabe411cbf0ecaaeef0c3b07
I20260812 19:04:15.971240 3718 ts_tablet_manager.cc:1939] T 7b317b74eabe411cbf0ecaaeef0c3b07 P 6183cb8c293e439daea1a4221ba83794: Deleting consensus metadata
I20260812 19:04:15.971541 3077 catalog_manager.cc:4841] TS 0f0aa9470f4b4e3aac661d0089845b9a (127.25.130.2:33077): tablet 7b317b74eabe411cbf0ecaaeef0c3b07 (table test-table [id=3b5e4376a17d44a18a730ceab3b2772e]) successfully deleted
I20260812 19:04:15.971871 3076 catalog_manager.cc:4841] TS 6183cb8c293e439daea1a4221ba83794 (127.25.130.3:46379): tablet 7b317b74eabe411cbf0ecaaeef0c3b07 (table test-table [id=3b5e4376a17d44a18a730ceab3b2772e]) successfully deleted
I20260812 19:04:15.972309 3719 ts_tablet_manager.cc:1905] T 7b317b74eabe411cbf0ecaaeef0c3b07 P d074369259f143bba8768daa96c9f43f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:04:15.974817 3719 ts_tablet_manager.cc:1918] T 7b317b74eabe411cbf0ecaaeef0c3b07 P d074369259f143bba8768daa96c9f43f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:04:15.974902 3719 log.cc:1196] T 7b317b74eabe411cbf0ecaaeef0c3b07 P d074369259f143bba8768daa96c9f43f: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-0/wal/wals/7b317b74eabe411cbf0ecaaeef0c3b07
I20260812 19:04:15.975144 3719 ts_tablet_manager.cc:1939] T 7b317b74eabe411cbf0ecaaeef0c3b07 P d074369259f143bba8768daa96c9f43f: Deleting consensus metadata
I20260812 19:04:15.975901 3077 catalog_manager.cc:4841] TS d074369259f143bba8768daa96c9f43f (127.25.130.1:37757): tablet 7b317b74eabe411cbf0ecaaeef0c3b07 (table test-table [id=3b5e4376a17d44a18a730ceab3b2772e]) successfully deleted
I20260812 19:04:15.975942 3094 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:04:15.976758 3094 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:04:15.981915 3415 tablet_service.cc:1467] Processing CreateTablet for tablet 8111466f2ccc473c9348a7a61aee32f9 (DEFAULT_TABLE table=test-table [id=4515cd0b269d40ec8525a780e4cc19f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:04:15.982265 3415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8111466f2ccc473c9348a7a61aee32f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:15.982263 3581 tablet_service.cc:1467] Processing CreateTablet for tablet 8111466f2ccc473c9348a7a61aee32f9 (DEFAULT_TABLE table=test-table [id=4515cd0b269d40ec8525a780e4cc19f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:04:15.982273 3250 tablet_service.cc:1467] Processing CreateTablet for tablet 8111466f2ccc473c9348a7a61aee32f9 (DEFAULT_TABLE table=test-table [id=4515cd0b269d40ec8525a780e4cc19f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:04:15.982534 3581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8111466f2ccc473c9348a7a61aee32f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:15.982560 3250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8111466f2ccc473c9348a7a61aee32f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:15.984869 3694 tablet_bootstrap.cc:492] T 8111466f2ccc473c9348a7a61aee32f9 P 6183cb8c293e439daea1a4221ba83794: Bootstrap starting.
I20260812 19:04:15.984997 3692 tablet_bootstrap.cc:492] T 8111466f2ccc473c9348a7a61aee32f9 P 0f0aa9470f4b4e3aac661d0089845b9a: Bootstrap starting.
I20260812 19:04:15.985894 3694 tablet_bootstrap.cc:654] T 8111466f2ccc473c9348a7a61aee32f9 P 6183cb8c293e439daea1a4221ba83794: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:15.985916 3692 tablet_bootstrap.cc:654] T 8111466f2ccc473c9348a7a61aee32f9 P 0f0aa9470f4b4e3aac661d0089845b9a: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:15.986984 3693 tablet_bootstrap.cc:492] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f: Bootstrap starting.
I20260812 19:04:15.987845 3693 tablet_bootstrap.cc:654] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:15.988721 3692 tablet_bootstrap.cc:492] T 8111466f2ccc473c9348a7a61aee32f9 P 0f0aa9470f4b4e3aac661d0089845b9a: No bootstrap required, opened a new log
I20260812 19:04:15.988879 3692 ts_tablet_manager.cc:1397] T 8111466f2ccc473c9348a7a61aee32f9 P 0f0aa9470f4b4e3aac661d0089845b9a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 19:04:15.989089 3694 tablet_bootstrap.cc:492] T 8111466f2ccc473c9348a7a61aee32f9 P 6183cb8c293e439daea1a4221ba83794: No bootstrap required, opened a new log
I20260812 19:04:15.989193 3694 ts_tablet_manager.cc:1397] T 8111466f2ccc473c9348a7a61aee32f9 P 6183cb8c293e439daea1a4221ba83794: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 19:04:15.989754 3694 raft_consensus.cc:348] T 8111466f2ccc473c9348a7a61aee32f9 P 6183cb8c293e439daea1a4221ba83794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.989881 3694 raft_consensus.cc:374] T 8111466f2ccc473c9348a7a61aee32f9 P 6183cb8c293e439daea1a4221ba83794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:15.989915 3694 raft_consensus.cc:729] T 8111466f2ccc473c9348a7a61aee32f9 P 6183cb8c293e439daea1a4221ba83794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6183cb8c293e439daea1a4221ba83794, State: Initialized, Role: FOLLOWER
I20260812 19:04:15.990046 3694 consensus_queue.cc:260] T 8111466f2ccc473c9348a7a61aee32f9 P 6183cb8c293e439daea1a4221ba83794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.990310 3694 ts_tablet_manager.cc:1428] T 8111466f2ccc473c9348a7a61aee32f9 P 6183cb8c293e439daea1a4221ba83794: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 19:04:15.990622 3692 raft_consensus.cc:348] T 8111466f2ccc473c9348a7a61aee32f9 P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.990826 3692 raft_consensus.cc:374] T 8111466f2ccc473c9348a7a61aee32f9 P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:15.990893 3692 raft_consensus.cc:729] T 8111466f2ccc473c9348a7a61aee32f9 P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f0aa9470f4b4e3aac661d0089845b9a, State: Initialized, Role: FOLLOWER
I20260812 19:04:15.991081 3692 consensus_queue.cc:260] T 8111466f2ccc473c9348a7a61aee32f9 P 0f0aa9470f4b4e3aac661d0089845b9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.991871 3692 ts_tablet_manager.cc:1428] T 8111466f2ccc473c9348a7a61aee32f9 P 0f0aa9470f4b4e3aac661d0089845b9a: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260812 19:04:15.992966 3693 tablet_bootstrap.cc:492] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f: No bootstrap required, opened a new log
I20260812 19:04:15.993073 3693 ts_tablet_manager.cc:1397] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260812 19:04:15.993642 3693 raft_consensus.cc:348] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.993772 3693 raft_consensus.cc:374] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:15.993803 3693 raft_consensus.cc:729] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d074369259f143bba8768daa96c9f43f, State: Initialized, Role: FOLLOWER
I20260812 19:04:15.993916 3693 consensus_queue.cc:260] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:15.994177 3693 ts_tablet_manager.cc:1428] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 19:04:16.175397 3700 raft_consensus.cc:482] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:04:16.175588 3700 raft_consensus.cc:504] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:16.176186 3700 leader_election.cc:288] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f0aa9470f4b4e3aac661d0089845b9a (127.25.130.2:33077), 6183cb8c293e439daea1a4221ba83794 (127.25.130.3:46379)
I20260812 19:04:16.176674 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8111466f2ccc473c9348a7a61aee32f9" candidate_uuid: "d074369259f143bba8768daa96c9f43f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" is_pre_election: true
I20260812 19:04:16.176771 3606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8111466f2ccc473c9348a7a61aee32f9" candidate_uuid: "d074369259f143bba8768daa96c9f43f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6183cb8c293e439daea1a4221ba83794" is_pre_election: true
I20260812 19:04:16.176846 3441 raft_consensus.cc:2454] T 8111466f2ccc473c9348a7a61aee32f9 P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d074369259f143bba8768daa96c9f43f in term 0.
I20260812 19:04:16.176928 3606 raft_consensus.cc:2454] T 8111466f2ccc473c9348a7a61aee32f9 P 6183cb8c293e439daea1a4221ba83794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d074369259f143bba8768daa96c9f43f in term 0.
I20260812 19:04:16.177300 3197 leader_election.cc:302] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f0aa9470f4b4e3aac661d0089845b9a, d074369259f143bba8768daa96c9f43f; no voters:
I20260812 19:04:16.177608 3700 raft_consensus.cc:2781] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:04:16.177685 3700 raft_consensus.cc:482] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:04:16.177731 3700 raft_consensus.cc:3037] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:16.179036 3700 raft_consensus.cc:504] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:16.179448 3700 leader_election.cc:288] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f [CANDIDATE]: Term 1 election: Requested vote from peers 0f0aa9470f4b4e3aac661d0089845b9a (127.25.130.2:33077), 6183cb8c293e439daea1a4221ba83794 (127.25.130.3:46379)
I20260812 19:04:16.179814 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8111466f2ccc473c9348a7a61aee32f9" candidate_uuid: "d074369259f143bba8768daa96c9f43f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f0aa9470f4b4e3aac661d0089845b9a"
I20260812 19:04:16.179844 3606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8111466f2ccc473c9348a7a61aee32f9" candidate_uuid: "d074369259f143bba8768daa96c9f43f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6183cb8c293e439daea1a4221ba83794"
I20260812 19:04:16.179955 3606 raft_consensus.cc:3037] T 8111466f2ccc473c9348a7a61aee32f9 P 6183cb8c293e439daea1a4221ba83794 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:16.179955 3441 raft_consensus.cc:3037] T 8111466f2ccc473c9348a7a61aee32f9 P 0f0aa9470f4b4e3aac661d0089845b9a [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:16.181066 3441 raft_consensus.cc:2454] T 8111466f2ccc473c9348a7a61aee32f9 P 0f0aa9470f4b4e3aac661d0089845b9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d074369259f143bba8768daa96c9f43f in term 1.
I20260812 19:04:16.181089 3606 raft_consensus.cc:2454] T 8111466f2ccc473c9348a7a61aee32f9 P 6183cb8c293e439daea1a4221ba83794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d074369259f143bba8768daa96c9f43f in term 1.
I20260812 19:04:16.181383 3196 leader_election.cc:302] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6183cb8c293e439daea1a4221ba83794, d074369259f143bba8768daa96c9f43f; no voters:
I20260812 19:04:16.181604 3700 raft_consensus.cc:2781] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:04:16.181740 3700 raft_consensus.cc:686] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f [term 1 LEADER]: Becoming Leader. State: Replica: d074369259f143bba8768daa96c9f43f, State: Running, Role: LEADER
I20260812 19:04:16.181923 3700 consensus_queue.cc:237] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } }
I20260812 19:04:16.183825 3091 catalog_manager.cc:5495] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f reported cstate change: term changed from 0 to 1, leader changed from <none> to d074369259f143bba8768daa96c9f43f (127.25.130.1). New cstate: current_term: 1 leader_uuid: "d074369259f143bba8768daa96c9f43f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d074369259f143bba8768daa96c9f43f" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 37757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 } health_report { overall_health: UNKNOWN } } }
I20260812 19:04:16.234266 3691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:16.231561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55128 (local address 127.25.130.1:37757)
0812 19:04:16.231684 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:16.231687 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:16.231700 (+ 13us) server_negotiation.cc:407] Connection header received
0812 19:04:16.231778 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:16.231780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:16.231822 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:16.231881 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:04:16.232046 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:16.232545 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:16.233362 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:16.233641 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:16.233725 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:16.233922 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:04:16.233991 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:04:16.234038 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:16.234094 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 19:04:16.234141 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 19:04:16.240402 3606 raft_consensus.cc:1261] T 8111466f2ccc473c9348a7a61aee32f9 P 6183cb8c293e439daea1a4221ba83794 [term 1 FOLLOWER]: Refusing update from remote peer d074369259f143bba8768daa96c9f43f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:04:16.240501 3441 raft_consensus.cc:1261] T 8111466f2ccc473c9348a7a61aee32f9 P 0f0aa9470f4b4e3aac661d0089845b9a [term 1 FOLLOWER]: Refusing update from remote peer d074369259f143bba8768daa96c9f43f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:04:16.240844 3713 consensus_queue.cc:1035] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6183cb8c293e439daea1a4221ba83794" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:04:16.241000 3700 consensus_queue.cc:1035] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f0aa9470f4b4e3aac661d0089845b9a" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:04:16.273415 3091 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:04:16.273620 3091 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:04:16.275588 3091 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b1beb356039e4f5185921f11c188d34b: Sending DeleteTablet for 3 replicas of tablet 8111466f2ccc473c9348a7a61aee32f9
I20260812 19:04:16.276156 3250 tablet_service.cc:1514] Processing DeleteTablet for tablet 8111466f2ccc473c9348a7a61aee32f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:04:16 UTC) from {username='oryx', principal='oryx/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:60240
I20260812 19:04:16.276268 3581 tablet_service.cc:1514] Processing DeleteTablet for tablet 8111466f2ccc473c9348a7a61aee32f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:04:16 UTC) from {username='oryx', principal='oryx/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:41036
I20260812 19:04:16.276240 3415 tablet_service.cc:1514] Processing DeleteTablet for tablet 8111466f2ccc473c9348a7a61aee32f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:04:16 UTC) from {username='oryx', principal='oryx/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:44490
I20260812 19:04:16.276367 3719 tablet_replica.cc:330] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f: stopping tablet replica
I20260812 19:04:16.276503 3720 tablet_replica.cc:330] T 8111466f2ccc473c9348a7a61aee32f9 P 0f0aa9470f4b4e3aac661d0089845b9a: stopping tablet replica
I20260812 19:04:16.276496 3719 raft_consensus.cc:2229] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:04:16.276612 3720 raft_consensus.cc:2229] T 8111466f2ccc473c9348a7a61aee32f9 P 0f0aa9470f4b4e3aac661d0089845b9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:04:16.276636 3718 tablet_replica.cc:330] T 8111466f2ccc473c9348a7a61aee32f9 P 6183cb8c293e439daea1a4221ba83794: stopping tablet replica
I20260812 19:04:16.276717 3720 raft_consensus.cc:2258] T 8111466f2ccc473c9348a7a61aee32f9 P 0f0aa9470f4b4e3aac661d0089845b9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:04:16.276717 3719 raft_consensus.cc:2258] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:04:16.276744 3718 raft_consensus.cc:2229] T 8111466f2ccc473c9348a7a61aee32f9 P 6183cb8c293e439daea1a4221ba83794 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:04:16.276844 3718 raft_consensus.cc:2258] T 8111466f2ccc473c9348a7a61aee32f9 P 6183cb8c293e439daea1a4221ba83794 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:04:16.277108 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 3178
I20260812 19:04:16.277426 3720 ts_tablet_manager.cc:1905] T 8111466f2ccc473c9348a7a61aee32f9 P 0f0aa9470f4b4e3aac661d0089845b9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:04:16.277488 3719 ts_tablet_manager.cc:1905] T 8111466f2ccc473c9348a7a61aee32f9 P d074369259f143bba8768daa96c9f43f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:04:16.279726 3718 ts_tablet_manager.cc:1905] T 8111466f2ccc473c9348a7a61aee32f9 P 6183cb8c293e439daea1a4221ba83794: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:04:16.279836 3720 ts_tablet_manager.cc:1918] T 8111466f2ccc473c9348a7a61aee32f9 P 0f0aa9470f4b4e3aac661d0089845b9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 19:04:16.279907 3720 log.cc:1196] T 8111466f2ccc473c9348a7a61aee32f9 P 0f0aa9470f4b4e3aac661d0089845b9a: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-1/wal/wals/8111466f2ccc473c9348a7a61aee32f9
I20260812 19:04:16.280149 3720 ts_tablet_manager.cc:1939] T 8111466f2ccc473c9348a7a61aee32f9 P 0f0aa9470f4b4e3aac661d0089845b9a: Deleting consensus metadata
I20260812 19:04:16.280872 3077 catalog_manager.cc:4841] TS 0f0aa9470f4b4e3aac661d0089845b9a (127.25.130.2:33077): tablet 8111466f2ccc473c9348a7a61aee32f9 (table test-table [id=4515cd0b269d40ec8525a780e4cc19f1]) successfully deleted
I20260812 19:04:16.282496 3718 ts_tablet_manager.cc:1918] T 8111466f2ccc473c9348a7a61aee32f9 P 6183cb8c293e439daea1a4221ba83794: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 19:04:16.282603 3718 log.cc:1196] T 8111466f2ccc473c9348a7a61aee32f9 P 6183cb8c293e439daea1a4221ba83794: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561415988404-26120-0/minicluster-data/ts-2/wal/wals/8111466f2ccc473c9348a7a61aee32f9
I20260812 19:04:16.282907 3718 ts_tablet_manager.cc:1939] T 8111466f2ccc473c9348a7a61aee32f9 P 6183cb8c293e439daea1a4221ba83794: Deleting consensus metadata
I20260812 19:04:16.283641 3076 catalog_manager.cc:4841] TS 6183cb8c293e439daea1a4221ba83794 (127.25.130.3:46379): tablet 8111466f2ccc473c9348a7a61aee32f9 (table test-table [id=4515cd0b269d40ec8525a780e4cc19f1]) successfully deleted
W20260812 19:04:16.289111 3077 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.130.1:37757 (error 108)
I20260812 19:04:16.289942 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 3338
I20260812 19:04:16.290256 3792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:16.289875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:37757 (local address 127.0.0.1:55138)
0812 19:04:16.290081 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:16.290158 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.1:37757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":52,"threads_started":1}
W20260812 19:04:16.291527 3077 catalog_manager.cc:4584] TS d074369259f143bba8768daa96c9f43f (127.25.130.1:37757): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8111466f2ccc473c9348a7a61aee32f9: Network error: Client connection negotiation failed: client connection to 127.25.130.1:37757: connect: Connection refused (error 111)
I20260812 19:04:16.299430 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 3506
I20260812 19:04:16.307808 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 3054
2026-08-12T19:04:16Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3260 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:04:16 dist-test-slave-49h3 krb5kdc[3795](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:04:16 dist-test-slave-49h3 krb5kdc[3795](info): set up 2 sockets
Aug 12 19:04:16 dist-test-slave-49h3 krb5kdc[3795](info): commencing operation
krb5kdc: starting...
W20260812 19:04:17.379424 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:04:17 dist-test-slave-49h3 krb5kdc[3795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:04:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:04:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:04:17.505923 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:38129
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:39979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:38129
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:04:17.619736 3809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:17.620047 3809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:17.620108 3809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:17.624202 3809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:04:17.624280 3809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:17.624307 3809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:17.624330 3809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:04:17.624351 3809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:04:17.629949 3809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:39979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:38129
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:38129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:17.631131 3809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:17.632217 3809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:17.640950 3817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:17.640959 3814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:17.640966 3815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:17.641242 3809 server_base.cc:1034] running on GCE node
I20260812 19:04:17.641969 3809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:17.643182 3809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:17.644382 3809 hybrid_clock.cc:648] HybridClock initialized: now 1786561457644361 us; error 34 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T19:04:17Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1342 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:04:17 dist-test-slave-49h3 krb5kdc[3822](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:04:17 dist-test-slave-49h3 krb5kdc[3822](info): set up 2 sockets
Aug 12 19:04:17 dist-test-slave-49h3 krb5kdc[3822](info): commencing operation
krb5kdc: starting...
W20260812 19:04:18.719841 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:04:18 dist-test-slave-49h3 krb5kdc[3822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:04:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:04:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:04:18.847823 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:33683
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:33683
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:04:18.963104 3839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:18.963610 3839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:18.963692 3839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:18.967991 3839 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:04:18.968084 3839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:18.968103 3839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:18.968119 3839 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:04:18.968134 3839 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:04:18.973862 3839 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:33683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:33683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:18.975047 3839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:18.976169 3839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:04:18.984980 3839 server_base.cc:1034] running on GCE node
W20260812 19:04:18.984941 3848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:18.984930 3844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:18.984941 3845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:18.985842 3839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:18.987068 3839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:18.988251 3839 hybrid_clock.cc:648] HybridClock initialized: now 1786561458988225 us; error 42 us; skew 500 ppm
Aug 12 19:04:18 dist-test-slave-49h3 krb5kdc[3822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561458, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:18.991076 3839 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:04:18.992063 3839 webserver.cc:460] Webserver started at http://127.25.130.62:46239/ using document root <none> and password file <none>
I20260812 19:04:18.992621 3839 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:18.992686 3839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:18.992864 3839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:18.995142 3839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "61e85af099ef447b9fbd7139c4032159"
format_stamp: "Formatted at 2026-08-12 19:04:18 on dist-test-slave-49h3"
server_key: "9dd0706e980d4ccaa0c1643c83d242b2"
server_key_iv: "86fc0331a0b5d6800e427988acd6e872"
server_key_version: "encryptionkey@0"
I20260812 19:04:18.995687 3839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "61e85af099ef447b9fbd7139c4032159"
format_stamp: "Formatted at 2026-08-12 19:04:18 on dist-test-slave-49h3"
server_key: "9dd0706e980d4ccaa0c1643c83d242b2"
server_key_iv: "86fc0331a0b5d6800e427988acd6e872"
server_key_version: "encryptionkey@0"
I20260812 19:04:18.999634 3839 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:04:19.002318 3855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:19.003618 3839 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:04:19.003753 3839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "61e85af099ef447b9fbd7139c4032159"
format_stamp: "Formatted at 2026-08-12 19:04:18 on dist-test-slave-49h3"
server_key: "9dd0706e980d4ccaa0c1643c83d242b2"
server_key_iv: "86fc0331a0b5d6800e427988acd6e872"
server_key_version: "encryptionkey@0"
I20260812 19:04:19.003859 3839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:19.044556 3839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:19.047777 3839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:19.047955 3839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:19.055182 3839 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:33683
I20260812 19:04:19.055243 3916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:33683 every 8 connection(s)
I20260812 19:04:19.056183 3839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:04:19.059031 3917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:19.064774 3917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159: Bootstrap starting.
I20260812 19:04:19.065667 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 3839
I20260812 19:04:19.065788 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:04:19.066083 26120 external_mini_cluster.cc:1428] Setting key b7fa5a44b22766e08aeb4e16a9f86898
I20260812 19:04:19.067628 3917 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:19.068389 3917 log.cc:824] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:19.070712 3917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159: No bootstrap required, opened a new log
Aug 12 19:04:19 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:04:19.075274 3917 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61e85af099ef447b9fbd7139c4032159" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 33683 } }
I20260812 19:04:19.075531 3917 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:19.075605 3917 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61e85af099ef447b9fbd7139c4032159, State: Initialized, Role: FOLLOWER
I20260812 19:04:19.076040 3917 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61e85af099ef447b9fbd7139c4032159" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 33683 } }
I20260812 19:04:19.076179 3917 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:04:19.076269 3917 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:04:19.076371 3917 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:19.077333 3917 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61e85af099ef447b9fbd7139c4032159" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 33683 } }
I20260812 19:04:19.077702 3917 leader_election.cc:302] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61e85af099ef447b9fbd7139c4032159; no voters:
I20260812 19:04:19.078024 3917 leader_election.cc:288] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:04:19.078132 3923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:04:19.078342 3923 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [term 1 LEADER]: Becoming Leader. State: Replica: 61e85af099ef447b9fbd7139c4032159, State: Running, Role: LEADER
I20260812 19:04:19.078711 3923 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61e85af099ef447b9fbd7139c4032159" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 33683 } }
I20260812 19:04:19.079437 3917 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:04:19.081321 3921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:19.067240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44286 (local address 127.25.130.62:33683)
0812 19:04:19.067733 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:19.067754 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:19.067798 (+ 44us) server_negotiation.cc:407] Connection header received
0812 19:04:19.068441 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:19.068457 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:19.068857 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:19.069229 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:19.070095 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:19.071129 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:19.072291 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:19.072767 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:19.072879 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:19.075757 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:19.075763 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:19.075778 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:19.075808 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:19.077591 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:19.078095 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:19.078102 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:19.078109 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:19.078312 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:19.078659 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:19.078663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:19.078665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:19.079121 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:19.079311 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:19.079970 (+ 659us) server_negotiation.cc:299] Negotiation successful
0812 19:04:19.080304 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":158,"threads_started":1}
I20260812 19:04:19.082219 3925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61e85af099ef447b9fbd7139c4032159. Latest consensus state: current_term: 1 leader_uuid: "61e85af099ef447b9fbd7139c4032159" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61e85af099ef447b9fbd7139c4032159" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 33683 } } }
I20260812 19:04:19.082356 3925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:19.083724 3924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61e85af099ef447b9fbd7139c4032159" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61e85af099ef447b9fbd7139c4032159" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 33683 } } }
I20260812 19:04:19.083838 3924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [sys.catalog]: This master's current role is: LEADER
W20260812 19:04:19.084668 3939 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:04:19.084740 3939 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:04:19.084829 3941 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:04:19.085613 3941 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:04:19.091465 3941 catalog_manager.cc:1347] Generated new cluster ID: 3f9e0e9f5c0744fbbe82cc205f71af6c
I20260812 19:04:19.091557 3941 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:04:19.101495 3941 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:04:19.102808 3941 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:04:19.110220 3941 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159: Generated new TSK 0
I20260812 19:04:19.110950 3941 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
I20260812 19:04:19.146696 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:0
--local_ip_for_outbound_sockets=127.25.130.1
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:33683
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:04:19.261154 3946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:19.261421 3946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:19.261500 3946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:19.265576 3946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:19.265646 3946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:19.265707 3946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:04:19.271147 3946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:33683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:19.272302 3946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:19.273401 3946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:19.283355 3954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:19.283372 3951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:19.283355 3952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:19.283592 3946 server_base.cc:1034] running on GCE node
I20260812 19:04:19.284080 3946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:19.284773 3946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:19.285928 3946 hybrid_clock.cc:648] HybridClock initialized: now 1786561459285894 us; error 51 us; skew 500 ppm
Aug 12 19:04:19 dist-test-slave-49h3 krb5kdc[3822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561459, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:19.288755 3946 init.cc:377] Logged in from keytab as kudu/127.25.130.1@KRBTEST.COM (short username kudu)
I20260812 19:04:19.289754 3946 webserver.cc:460] Webserver started at http://127.25.130.1:45365/ using document root <none> and password file <none>
I20260812 19:04:19.290307 3946 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:19.290374 3946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:19.290601 3946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:19.292745 3946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/data/instance:
uuid: "1722dd735afc439089a023165b789306"
format_stamp: "Formatted at 2026-08-12 19:04:19 on dist-test-slave-49h3"
server_key: "ce4116bfbfbd4752f7f720f846878d68"
server_key_iv: "a3421b51d34047533dd909f4fd43ef8d"
server_key_version: "encryptionkey@0"
I20260812 19:04:19.293341 3946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance:
uuid: "1722dd735afc439089a023165b789306"
format_stamp: "Formatted at 2026-08-12 19:04:19 on dist-test-slave-49h3"
server_key: "ce4116bfbfbd4752f7f720f846878d68"
server_key_iv: "a3421b51d34047533dd909f4fd43ef8d"
server_key_version: "encryptionkey@0"
I20260812 19:04:19.297417 3946 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 19:04:19.300209 3961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:19.301559 3946 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:04:19.301682 3946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "1722dd735afc439089a023165b789306"
format_stamp: "Formatted at 2026-08-12 19:04:19 on dist-test-slave-49h3"
server_key: "ce4116bfbfbd4752f7f720f846878d68"
server_key_iv: "a3421b51d34047533dd909f4fd43ef8d"
server_key_version: "encryptionkey@0"
I20260812 19:04:19.301789 3946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:19.329037 3946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:19.332026 3946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:19.332201 3946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:19.332788 3946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:19.333726 3946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:19.333781 3946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:19.333823 3946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:19.333843 3946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:19.342886 3946 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:43407
I20260812 19:04:19.342911 4087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:43407 every 8 connection(s)
I20260812 19:04:19.343834 3946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:04:19.344282 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 3946
I20260812 19:04:19.344420 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance
I20260812 19:04:19.344741 26120 external_mini_cluster.cc:1428] Setting key e46b3c9595976d78dddd0ad26cada742
Aug 12 19:04:19 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561459, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.130.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.2@KRBTEST.COM" created.
I20260812 19:04:19.359596 3921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:19.345796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:57599 (local address 127.25.130.62:33683)
0812 19:04:19.345937 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:19.345942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:19.346941 (+ 999us) server_negotiation.cc:407] Connection header received
0812 19:04:19.348318 (+ 1377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:19.348321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:19.348380 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:19.348461 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:19.350103 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:19.350644 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:19.352389 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:19.352768 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:19.352907 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:19.355726 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:19.355730 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:19.355732 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:19.355761 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:19.357054 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:19.357697 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:19.357702 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:19.357703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:19.357843 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:19.358169 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:19.358171 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:19.358172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:19.358457 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:19.358542 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:19.359267 (+ 725us) server_negotiation.cc:299] Negotiation successful
0812 19:04:19.359420 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 19:04:19.360375 4090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:19.346276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.1:57599)
0812 19:04:19.346798 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:19.346843 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:19.348078 (+ 1235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:19.348654 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:19.348663 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:19.349168 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:19.349895 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:19.349915 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:19.350790 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:19.350793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:19.352230 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:19.352243 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:19.352931 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:19.352933 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:19.353088 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:19.353764 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:19.353783 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:19.355495 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:19.357197 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:19.357203 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:19.357219 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:19.357602 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:19.357926 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:19.357928 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:19.357929 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:19.358090 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:19.358576 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:19.358584 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:19.358950 (+ 366us) client_negotiation.cc:769] Sending connection context
0812 19:04:19.359150 (+ 200us) client_negotiation.cc:240] Negotiation successful
0812 19:04:19.359455 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":155,"threads_started":1}
I20260812 19:04:19.361869 4088 heartbeater.cc:347] Connected to a master server at 127.25.130.62:33683
I20260812 19:04:19.362172 4088 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:19.362797 4088 heartbeater.cc:511] Master 127.25.130.62:33683 requested a full tablet report, sending...
I20260812 19:04:19.364473 3874 ts_manager.cc:194] Registered new tserver with Master: 1722dd735afc439089a023165b789306 (127.25.130.1:43407)
I20260812 19:04:19.365721 3874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.1@KRBTEST.COM'} at 127.25.130.1:57599
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
I20260812 19:04:19.375824 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:0
--local_ip_for_outbound_sockets=127.25.130.2
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:33683
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:04:19.491104 4093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:19.491379 4093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:19.491425 4093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:19.495558 4093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:19.495631 4093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:19.495702 4093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:04:19.501214 4093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:33683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:19.502521 4093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:19.503640 4093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:19.513846 4098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:19.513896 4093 server_base.cc:1034] running on GCE node
W20260812 19:04:19.513849 4099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:19.513846 4101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:19.514503 4093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:19.515141 4093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:19.516295 4093 hybrid_clock.cc:648] HybridClock initialized: now 1786561459516267 us; error 42 us; skew 500 ppm
Aug 12 19:04:19 dist-test-slave-49h3 krb5kdc[3822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561459, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:19.519119 4093 init.cc:377] Logged in from keytab as kudu/127.25.130.2@KRBTEST.COM (short username kudu)
I20260812 19:04:19.520117 4093 webserver.cc:460] Webserver started at http://127.25.130.2:33277/ using document root <none> and password file <none>
I20260812 19:04:19.520723 4093 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:19.520799 4093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:19.521032 4093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:19.523373 4093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/data/instance:
uuid: "7e30fb1819fe4c7f9951881ccaa716df"
format_stamp: "Formatted at 2026-08-12 19:04:19 on dist-test-slave-49h3"
server_key: "25f7a4b9d9c6348b0b386c86f5131c65"
server_key_iv: "ac1c79b84ec4773a303a802fea1eec77"
server_key_version: "encryptionkey@0"
I20260812 19:04:19.524089 4093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance:
uuid: "7e30fb1819fe4c7f9951881ccaa716df"
format_stamp: "Formatted at 2026-08-12 19:04:19 on dist-test-slave-49h3"
server_key: "25f7a4b9d9c6348b0b386c86f5131c65"
server_key_iv: "ac1c79b84ec4773a303a802fea1eec77"
server_key_version: "encryptionkey@0"
I20260812 19:04:19.528189 4093 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260812 19:04:19.530853 4108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:19.532167 4093 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 19:04:19.532299 4093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "7e30fb1819fe4c7f9951881ccaa716df"
format_stamp: "Formatted at 2026-08-12 19:04:19 on dist-test-slave-49h3"
server_key: "25f7a4b9d9c6348b0b386c86f5131c65"
server_key_iv: "ac1c79b84ec4773a303a802fea1eec77"
server_key_version: "encryptionkey@0"
I20260812 19:04:19.532414 4093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:19.544581 4093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:19.547518 4093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:19.547712 4093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:19.548314 4093 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:19.549161 4093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:19.549223 4093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:19.549278 4093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:19.549302 4093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:19.558106 4093 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:43425
I20260812 19:04:19.558140 4232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:43425 every 8 connection(s)
I20260812 19:04:19.559039 4093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:04:19.563548 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 4093
I20260812 19:04:19.563676 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance
I20260812 19:04:19.563944 26120 external_mini_cluster.cc:1428] Setting key 0fdd8e93f3ec1ea1211246acdf39364f
Aug 12 19:04:19 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561459, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:04:19.573012 3921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:19.560857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:59569 (local address 127.25.130.62:33683)
0812 19:04:19.561034 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:19.561039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:19.561924 (+ 885us) server_negotiation.cc:407] Connection header received
0812 19:04:19.563186 (+ 1262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:19.563189 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:19.563237 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:19.563326 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:19.564551 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:19.565260 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:19.566447 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:19.566661 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:19.566763 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:19.569489 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:19.569493 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:19.569496 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:19.569523 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:19.570819 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:19.571441 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:19.571443 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:19.571444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:19.571517 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:19.571888 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:19.571890 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:19.571891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:19.572067 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:19.572137 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:19.572778 (+ 641us) server_negotiation.cc:299] Negotiation successful
0812 19:04:19.572878 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 19:04:19.574327 4235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:19.561271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.2:59569)
0812 19:04:19.561767 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:19.561812 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:19.562986 (+ 1174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:19.563481 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:19.563488 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:19.563877 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:19.564402 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:19.564417 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:19.565397 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:19.565399 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:19.566326 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:19.566336 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:19.566817 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:19.566821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:19.566981 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:19.567580 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:19.567602 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:19.569290 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:19.570946 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:19.570953 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:19.570972 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:19.571342 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:19.571596 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:19.571599 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:19.571601 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:19.571805 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:19.572163 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:19.572170 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:19.572514 (+ 344us) client_negotiation.cc:769] Sending connection context
0812 19:04:19.572758 (+ 244us) client_negotiation.cc:240] Negotiation successful
0812 19:04:19.573135 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":151,"threads_started":1}
WARNING: no policy specified for kudu/127.25.130.3@KRBTEST.COM; defaulting to no policy
I20260812 19:04:19.575531 4233 heartbeater.cc:347] Connected to a master server at 127.25.130.62:33683
I20260812 19:04:19.575768 4233 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:19.576277 4233 heartbeater.cc:511] Master 127.25.130.62:33683 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.3@KRBTEST.COM" created.
I20260812 19:04:19.577359 3874 ts_manager.cc:194] Registered new tserver with Master: 7e30fb1819fe4c7f9951881ccaa716df (127.25.130.2:43425)
I20260812 19:04:19.577967 3874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.2@KRBTEST.COM'} at 127.25.130.2:59569
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
I20260812 19:04:19.593055 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:0
--local_ip_for_outbound_sockets=127.25.130.3
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:33683
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:04:19.707253 4238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:19.707551 4238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:19.707595 4238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:19.711987 4238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:19.712064 4238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:19.712126 4238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:04:19.717528 4238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:33683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:19.718586 4238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:19.719650 4238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:19.729256 4245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:19.729259 4248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:19.729203 4246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:19.729538 4238 server_base.cc:1034] running on GCE node
I20260812 19:04:19.729955 4238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:19.730609 4238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:19.731760 4238 hybrid_clock.cc:648] HybridClock initialized: now 1786561459731735 us; error 42 us; skew 500 ppm
Aug 12 19:04:19 dist-test-slave-49h3 krb5kdc[3822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561459, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:19.734525 4238 init.cc:377] Logged in from keytab as kudu/127.25.130.3@KRBTEST.COM (short username kudu)
I20260812 19:04:19.735419 4238 webserver.cc:460] Webserver started at http://127.25.130.3:42615/ using document root <none> and password file <none>
I20260812 19:04:19.736079 4238 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:19.736145 4238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:19.736325 4238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:19.738511 4238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/data/instance:
uuid: "823cc4f6945d42d5ac1fce8b2aff9ea4"
format_stamp: "Formatted at 2026-08-12 19:04:19 on dist-test-slave-49h3"
server_key: "45972a8480c225ebdc319380947b0ed8"
server_key_iv: "1c26a5506c67221ebf434c8309583295"
server_key_version: "encryptionkey@0"
I20260812 19:04:19.739063 4238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance:
uuid: "823cc4f6945d42d5ac1fce8b2aff9ea4"
format_stamp: "Formatted at 2026-08-12 19:04:19 on dist-test-slave-49h3"
server_key: "45972a8480c225ebdc319380947b0ed8"
server_key_iv: "1c26a5506c67221ebf434c8309583295"
server_key_version: "encryptionkey@0"
I20260812 19:04:19.743019 4238 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:04:19.745568 4257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:19.746893 4238 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:04:19.747010 4238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "823cc4f6945d42d5ac1fce8b2aff9ea4"
format_stamp: "Formatted at 2026-08-12 19:04:19 on dist-test-slave-49h3"
server_key: "45972a8480c225ebdc319380947b0ed8"
server_key_iv: "1c26a5506c67221ebf434c8309583295"
server_key_version: "encryptionkey@0"
I20260812 19:04:19.747125 4238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:19.766152 4238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:19.769480 4238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:19.769685 4238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:19.770311 4238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:19.771203 4238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:19.771255 4238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:19.771311 4238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:19.771338 4238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:19.780022 4238 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:37307
I20260812 19:04:19.780059 4388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:37307 every 8 connection(s)
I20260812 19:04:19.781003 4238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
Aug 12 19:04:19 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561459, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:04:19.790706 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 4238
I20260812 19:04:19.790858 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance
I20260812 19:04:19.791191 26120 external_mini_cluster.cc:1428] Setting key 6fbd00aeaae80fc1f61bb9aabe5124f2
I20260812 19:04:19.795171 3921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:19.782907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:34989 (local address 127.25.130.62:33683)
0812 19:04:19.783071 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:19.783075 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:19.783905 (+ 830us) server_negotiation.cc:407] Connection header received
0812 19:04:19.785219 (+ 1314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:19.785222 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:19.785271 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:19.785344 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:19.786566 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:19.787097 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:19.788166 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:19.788369 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:19.788435 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:19.791106 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:19.791111 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:19.791114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:19.791146 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:19.792903 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:19.793532 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:19.793536 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:19.793538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:19.793643 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:19.794083 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:19.794087 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:19.794089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:19.794309 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:19.794389 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:19.794931 (+ 542us) server_negotiation.cc:299] Negotiation successful
0812 19:04:19.795043 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 19:04:19.796108 4393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:19.783288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.3:34989)
0812 19:04:19.783745 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:19.783791 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:19.784998 (+ 1207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:19.785543 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:19.785552 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:19.785915 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:19.786441 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:19.786455 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:19.787193 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:19.787195 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:19.788075 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:19.788084 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:19.788479 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:19.788482 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:19.788622 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:19.789198 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:19.789215 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:19.790886 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:19.793065 (+ 2179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:19.793073 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:19.793094 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:19.793414 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:19.793747 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:19.793751 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:19.793752 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:19.793975 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:19.794412 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:19.794418 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:19.794720 (+ 302us) client_negotiation.cc:769] Sending connection context
0812 19:04:19.794885 (+ 165us) client_negotiation.cc:240] Negotiation successful
0812 19:04:19.795182 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":118,"threads_started":1}
I20260812 19:04:19.797312 4389 heartbeater.cc:347] Connected to a master server at 127.25.130.62:33683
I20260812 19:04:19.797600 4389 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:19.798132 4389 heartbeater.cc:511] Master 127.25.130.62:33683 requested a full tablet report, sending...
I20260812 19:04:19.799223 3874 ts_manager.cc:194] Registered new tserver with Master: 823cc4f6945d42d5ac1fce8b2aff9ea4 (127.25.130.3:37307)
I20260812 19:04:19.799885 3874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.3@KRBTEST.COM'} at 127.25.130.3:34989
I20260812 19:04:19.805934 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.25.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.1.keytab.
Entry for principal oryx/127.25.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.1.keytab.
WARNING: no policy specified for oryx/127.25.130.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.130.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.2.keytab.
Entry for principal oryx/127.25.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.2.keytab.
WARNING: no policy specified for oryx/127.25.130.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.3.keytab.
Entry for principal oryx/127.25.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.3.keytab.
I20260812 19:04:19.886195 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 3946
I20260812 19:04:19.898595 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 4093
I20260812 19:04:19.906174 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 4238
I20260812 19:04:19.914623 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 3839
I20260812 19:04:19.923470 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:33683
--webserver_interface=127.25.130.62
--webserver_port=46239
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:33683
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:04:20.040558 4401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:20.040834 4401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:20.040880 4401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:20.044961 4401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:04:20.045037 4401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:20.045054 4401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:20.045071 4401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:04:20.045100 4401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:04:20.050804 4401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:33683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:33683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=46239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:20.051926 4401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:20.052986 4401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:20.061538 4409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:20.061641 4407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:20.061856 4401 server_base.cc:1034] running on GCE node
W20260812 19:04:20.061542 4406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:20.062661 4401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:20.063833 4401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:20.064989 4401 hybrid_clock.cc:648] HybridClock initialized: now 1786561460064966 us; error 39 us; skew 500 ppm
Aug 12 19:04:20 dist-test-slave-49h3 krb5kdc[3822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561460, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:20.067863 4401 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:04:20.068890 4401 webserver.cc:460] Webserver started at http://127.25.130.62:46239/ using document root <none> and password file <none>
I20260812 19:04:20.069506 4401 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:20.069597 4401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:20.073704 4401 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 19:04:20.075956 4418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:20.077282 4401 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:04:20.077401 4401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "61e85af099ef447b9fbd7139c4032159"
format_stamp: "Formatted at 2026-08-12 19:04:18 on dist-test-slave-49h3"
server_key: "9dd0706e980d4ccaa0c1643c83d242b2"
server_key_iv: "86fc0331a0b5d6800e427988acd6e872"
server_key_version: "encryptionkey@0"
I20260812 19:04:20.078038 4401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:20.104079 4401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:20.107424 4401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:20.107620 4401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:20.115271 4401 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:33683
I20260812 19:04:20.115298 4481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:33683 every 8 connection(s)
I20260812 19:04:20.116374 4401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:04:20.120530 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 4401
I20260812 19:04:20.120851 4482 sys_catalog.cc:263] Verifying existing consensus state
I20260812 19:04:20.120995 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:43407
--local_ip_for_outbound_sockets=127.25.130.1
--tserver_master_addrs=127.25.130.62:33683
--webserver_port=45365
--webserver_interface=127.25.130.1
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.25.130.1
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:04:20.122855 4482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159: Bootstrap starting.
I20260812 19:04:20.141057 4482 log.cc:824] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:20.147894 4482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:04:20.148268 4482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159: Bootstrap complete.
I20260812 19:04:20.152832 4482 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61e85af099ef447b9fbd7139c4032159" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 33683 } }
I20260812 19:04:20.153308 4482 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61e85af099ef447b9fbd7139c4032159, State: Initialized, Role: FOLLOWER
I20260812 19:04:20.153867 4482 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61e85af099ef447b9fbd7139c4032159" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 33683 } }
I20260812 19:04:20.154006 4482 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:04:20.154085 4482 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:04:20.154184 4482 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:04:20.156288 4482 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61e85af099ef447b9fbd7139c4032159" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 33683 } }
I20260812 19:04:20.156628 4482 leader_election.cc:302] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61e85af099ef447b9fbd7139c4032159; no voters:
I20260812 19:04:20.156992 4482 leader_election.cc:288] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 19:04:20.157109 4486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:04:20.157327 4486 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [term 2 LEADER]: Becoming Leader. State: Replica: 61e85af099ef447b9fbd7139c4032159, State: Running, Role: LEADER
I20260812 19:04:20.157678 4486 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61e85af099ef447b9fbd7139c4032159" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 33683 } }
I20260812 19:04:20.158380 4482 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:04:20.158705 4486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61e85af099ef447b9fbd7139c4032159. Latest consensus state: current_term: 2 leader_uuid: "61e85af099ef447b9fbd7139c4032159" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61e85af099ef447b9fbd7139c4032159" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 33683 } } }
I20260812 19:04:20.158830 4486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:20.159511 4487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "61e85af099ef447b9fbd7139c4032159" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61e85af099ef447b9fbd7139c4032159" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 33683 } } }
I20260812 19:04:20.159691 4487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159 [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:20.160022 4493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:04:20.163446 4493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:04:20.164350 4493 catalog_manager.cc:1259] Loaded cluster ID: 3f9e0e9f5c0744fbbe82cc205f71af6c
I20260812 19:04:20.164412 4493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:04:20.166821 4493 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:04:20.167562 4493 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 61e85af099ef447b9fbd7139c4032159: Loaded TSK: 0
I20260812 19:04:20.168969 4493 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 19:04:20.239367 4484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:20.239660 4484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:20.239707 4484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:20.243885 4484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:20.243971 4484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:20.244073 4484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:04:20.249804 4484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.1.keytab
--principal=oryx/127.25.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:43407
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=45365
--tserver_master_addrs=127.25.130.62:33683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:20.250965 4484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:20.252027 4484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:04:20.262070 4484 server_base.cc:1034] running on GCE node
W20260812 19:04:20.261981 4516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:20.261981 4518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:20.262326 4515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:20.262791 4484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:20.263464 4484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:20.264637 4484 hybrid_clock.cc:648] HybridClock initialized: now 1786561460264601 us; error 57 us; skew 500 ppm
Aug 12 19:04:20 dist-test-slave-49h3 krb5kdc[3822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561460, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:20.267439 4484 init.cc:377] Logged in from keytab as oryx/127.25.130.1@KRBTEST.COM (short username oryx)
I20260812 19:04:20.268455 4484 webserver.cc:460] Webserver started at http://127.25.130.1:45365/ using document root <none> and password file <none>
I20260812 19:04:20.269022 4484 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:20.269102 4484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:20.273334 4484 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:04:20.275748 4527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:20.277127 4484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 19:04:20.277256 4484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "1722dd735afc439089a023165b789306"
format_stamp: "Formatted at 2026-08-12 19:04:19 on dist-test-slave-49h3"
server_key: "ce4116bfbfbd4752f7f720f846878d68"
server_key_iv: "a3421b51d34047533dd909f4fd43ef8d"
server_key_version: "encryptionkey@0"
I20260812 19:04:20.277882 4484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:20.310061 4484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:20.313412 4484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:20.313679 4484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:20.314381 4484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:20.315224 4484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:20.315274 4484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:20.315315 4484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:20.315335 4484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:20.324292 4484 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:43407
I20260812 19:04:20.324321 4659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:43407 every 8 connection(s)
I20260812 19:04:20.325361 4484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:04:20.329284 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 4484
I20260812 19:04:20.329841 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:43425
--local_ip_for_outbound_sockets=127.25.130.2
--tserver_master_addrs=127.25.130.62:33683
--webserver_port=33277
--webserver_interface=127.25.130.2
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.25.130.2
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 19:04:20 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:20 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:20.339932 4663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:20.327766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.1:54705)
0812 19:04:20.328288 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:20.328337 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:20.329859 (+ 1522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:20.331101 (+ 1242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:20.331110 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:20.331631 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:20.332121 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:20.332135 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.333941 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.333944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:20.334912 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:20.334924 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.336020 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.336024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:20.336187 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:20.336786 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:20.336804 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:20.339119 (+ 2315us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":121,"threads_started":1}
W20260812 19:04:20.340528 4660 heartbeater.cc:644] Failed to heartbeat to 127.25.130.62:33683 (0 consecutive failures): Not authorized: Failed to ping master at 127.25.130.62:33683: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
W20260812 19:04:20.341027 4662 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:20.327648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:54705 (local address 127.25.130.62:33683)
0812 19:04:20.328224 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:20.328244 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:20.328442 (+ 198us) server_negotiation.cc:407] Connection header received
0812 19:04:20.330329 (+ 1887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:20.330341 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:20.330664 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:20.330952 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:20.332313 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.333810 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.335054 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.335816 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.336008 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:20.339446 (+ 3438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.1:54705: BlockingRecv error: recv got EOF from 127.25.130.1:54705 (error 108)
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":147,"threads_started":1}
Aug 12 19:04:20 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:20 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:20.346751 4663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:20.341059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.1:39357)
0812 19:04:20.341242 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:20.341257 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:20.341340 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:20.341879 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:20.341882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:20.342024 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:20.342132 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:20.342138 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.343151 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.343153 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:20.343710 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:20.343718 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.344282 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.344285 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:20.344363 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:20.344836 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:20.344848 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:20.346613 (+ 1765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113}
W20260812 19:04:20.347148 4662 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:20.341167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:39357 (local address 127.25.130.62:33683)
0812 19:04:20.341641 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:20.341646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:20.341661 (+ 15us) server_negotiation.cc:407] Connection header received
0812 19:04:20.341702 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:20.341705 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:20.341754 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:20.341831 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:20.342235 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.343048 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.343808 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.344157 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.344261 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:20.346764 (+ 2503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.1:39357: BlockingRecv error: recv got EOF from 127.25.130.1:39357 (error 108)
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":128,"threads_started":1}
Aug 12 19:04:20 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:20 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:20.352319 4663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:20.347330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.1:38899)
0812 19:04:20.347492 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:20.347505 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:20.347587 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:20.347792 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:20.347794 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:20.347917 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:20.347997 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:20.348001 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.348997 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.348999 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:20.349611 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:20.349621 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.350154 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.350156 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:20.350210 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:20.350635 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:20.350647 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:20.352216 (+ 1569us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96}
W20260812 19:04:20.352502 4662 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:20.347456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:38899 (local address 127.25.130.62:33683)
0812 19:04:20.347574 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:20.347579 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:20.347594 (+ 15us) server_negotiation.cc:407] Connection header received
0812 19:04:20.347653 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:20.347656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:20.347701 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:20.347780 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:20.348083 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.348887 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.349715 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.350058 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.350160 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:20.352387 (+ 2227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.1:38899: BlockingRecv error: recv got EOF from 127.25.130.1:38899 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260812 19:04:20.352558 4660 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 19:04:20.448993 4665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:20.449283 4665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:20.449328 4665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:20.453397 4665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:20.453506 4665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:20.453576 4665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:04:20.459228 4665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.2.keytab
--principal=oryx/127.25.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:43425
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=33277
--tserver_master_addrs=127.25.130.62:33683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:20.460321 4665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:20.461346 4665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:20.471058 4673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:20.471117 4665 server_base.cc:1034] running on GCE node
W20260812 19:04:20.471031 4676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:20.471017 4674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:20.471699 4665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:20.472384 4665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:20.473526 4665 hybrid_clock.cc:648] HybridClock initialized: now 1786561460473507 us; error 36 us; skew 500 ppm
Aug 12 19:04:20 dist-test-slave-49h3 krb5kdc[3822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561460, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:20.476285 4665 init.cc:377] Logged in from keytab as oryx/127.25.130.2@KRBTEST.COM (short username oryx)
I20260812 19:04:20.477226 4665 webserver.cc:460] Webserver started at http://127.25.130.2:33277/ using document root <none> and password file <none>
I20260812 19:04:20.477840 4665 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:20.477927 4665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:20.481950 4665 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 19:04:20.484210 4684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:20.485548 4665 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:04:20.485677 4665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "7e30fb1819fe4c7f9951881ccaa716df"
format_stamp: "Formatted at 2026-08-12 19:04:19 on dist-test-slave-49h3"
server_key: "25f7a4b9d9c6348b0b386c86f5131c65"
server_key_iv: "ac1c79b84ec4773a303a802fea1eec77"
server_key_version: "encryptionkey@0"
I20260812 19:04:20.486275 4665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:20.500703 4665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:20.504029 4665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:20.504220 4665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:20.504844 4665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:20.505739 4665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:20.505800 4665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:20.505843 4665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:20.505872 4665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:20.514704 4665 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:43425
I20260812 19:04:20.514735 4812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:43425 every 8 connection(s)
I20260812 19:04:20.515717 4665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:04:20.517956 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 4665
I20260812 19:04:20.518414 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:37307
--local_ip_for_outbound_sockets=127.25.130.3
--tserver_master_addrs=127.25.130.62:33683
--webserver_port=42615
--webserver_interface=127.25.130.3
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.25.130.3
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 19:04:20 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:20 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
W20260812 19:04:20.528558 4662 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:20.517639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:44049 (local address 127.25.130.62:33683)
0812 19:04:20.517812 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:20.517817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:20.518823 (+ 1006us) server_negotiation.cc:407] Connection header received
0812 19:04:20.520565 (+ 1742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:20.520570 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:20.520637 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:20.520742 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:20.522035 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.522796 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.524033 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.524378 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.524506 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:20.528410 (+ 3904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.2:44049: BlockingRecv error: recv got EOF from 127.25.130.2:44049 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 19:04:20.529181 4815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:20.518102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.2:44049)
0812 19:04:20.518643 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:20.518704 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:20.520298 (+ 1594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:20.520897 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:20.520904 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:20.521307 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:20.521866 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:20.521882 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.522920 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.522922 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:20.523887 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:20.523898 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.524576 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.524580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:20.524753 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:20.525501 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:20.525527 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:20.528206 (+ 2679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":128,"threads_started":1}
W20260812 19:04:20.529855 4813 heartbeater.cc:644] Failed to heartbeat to 127.25.130.62:33683 (0 consecutive failures): Not authorized: Failed to ping master at 127.25.130.62:33683: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Aug 12 19:04:20 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:20 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:20.535466 4815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:20.530355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.2:45177)
0812 19:04:20.530459 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:20.530473 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:20.530557 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:20.530802 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:20.530804 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:20.530959 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:20.531064 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:20.531069 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.531798 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.531801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:20.532362 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:20.532371 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.532801 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.532805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:20.532897 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:20.533416 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:20.533431 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:20.535334 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 19:04:20.535602 4662 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:20.530472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:45177 (local address 127.25.130.62:33683)
0812 19:04:20.530603 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:20.530607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:20.530621 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:04:20.530667 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:20.530669 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:20.530706 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:20.530768 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:20.531176 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.531693 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.532476 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.532697 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.532783 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:20.535495 (+ 2712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.2:45177: BlockingRecv error: recv got EOF from 127.25.130.2:45177 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 12 19:04:20 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:20 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:20.541513 4815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:20.536457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.2:36921)
0812 19:04:20.536628 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:20.536642 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:20.536720 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:20.536937 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:20.536940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:20.537089 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:20.537164 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:20.537169 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.537923 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.537927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:20.538436 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:20.538444 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.538864 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.538868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:20.538944 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:20.539479 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:20.539492 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:20.541341 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106}
W20260812 19:04:20.541656 4662 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:20.536567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:36921 (local address 127.25.130.62:33683)
0812 19:04:20.536683 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:20.536687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:20.536700 (+ 13us) server_negotiation.cc:407] Connection header received
0812 19:04:20.536804 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:20.536806 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:20.536845 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:20.536906 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:20.537270 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.537819 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.538544 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.538755 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.538850 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:20.541547 (+ 2697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.2:36921: BlockingRecv error: recv got EOF from 127.25.130.2:36921 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260812 19:04:20.541989 4813 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 19:04:20.634907 4817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:20.635211 4817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:20.635254 4817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:20.639402 4817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:20.639484 4817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:20.639549 4817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:04:20.645177 4817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/krb5kdc/oryx_127.25.130.3.keytab
--principal=oryx/127.25.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:37307
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=42615
--tserver_master_addrs=127.25.130.62:33683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:20.646382 4817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:20.647476 4817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:20.657307 4824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:20.657307 4825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:20.657301 4827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:20.657622 4817 server_base.cc:1034] running on GCE node
I20260812 19:04:20.658057 4817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:20.658881 4817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:20.660071 4817 hybrid_clock.cc:648] HybridClock initialized: now 1786561460660042 us; error 38 us; skew 500 ppm
Aug 12 19:04:20 dist-test-slave-49h3 krb5kdc[3822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561460, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:20.662997 4817 init.cc:377] Logged in from keytab as oryx/127.25.130.3@KRBTEST.COM (short username oryx)
I20260812 19:04:20.663973 4817 webserver.cc:460] Webserver started at http://127.25.130.3:42615/ using document root <none> and password file <none>
I20260812 19:04:20.664562 4817 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:20.664644 4817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:20.668632 4817 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 19:04:20.671206 4836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:20.672494 4817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 19:04:20.672605 4817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "823cc4f6945d42d5ac1fce8b2aff9ea4"
format_stamp: "Formatted at 2026-08-12 19:04:19 on dist-test-slave-49h3"
server_key: "45972a8480c225ebdc319380947b0ed8"
server_key_iv: "1c26a5506c67221ebf434c8309583295"
server_key_version: "encryptionkey@0"
I20260812 19:04:20.673175 4817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:20.692538 4817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:20.695858 4817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:20.696050 4817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:20.696686 4817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:20.697602 4817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:20.697664 4817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:20.697712 4817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:20.697742 4817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:20.706336 4817 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:37307
I20260812 19:04:20.706364 4966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:37307 every 8 connection(s)
I20260812 19:04:20.707376 4817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
Aug 12 19:04:20 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:20.716848 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 4817
Aug 12 19:04:20 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
W20260812 19:04:20.718688 4662 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:20.709145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:52605 (local address 127.25.130.62:33683)
0812 19:04:20.709275 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:20.709279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:20.710169 (+ 890us) server_negotiation.cc:407] Connection header received
0812 19:04:20.711487 (+ 1318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:20.711491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:20.711561 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:20.711641 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:20.712853 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.713379 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.714646 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.714886 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.714976 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:20.718545 (+ 3569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.3:52605: BlockingRecv error: recv got EOF from 127.25.130.3:52605 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 19:04:20.719322 4970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:20.709538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.3:52605)
0812 19:04:20.709997 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:20.710040 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:20.711263 (+ 1223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:20.711832 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:20.711839 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:20.712217 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:20.712694 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:20.712708 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.713537 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.713540 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:20.714490 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:20.714503 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.715022 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.715025 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:20.715180 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:20.715694 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:20.715710 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:20.718389 (+ 2679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":123,"threads_started":1}
W20260812 19:04:20.719835 4967 heartbeater.cc:644] Failed to heartbeat to 127.25.130.62:33683 (0 consecutive failures): Not authorized: Failed to ping master at 127.25.130.62:33683: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Aug 12 19:04:20 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:20 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:20.725147 4970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:20.720252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.3:47961)
0812 19:04:20.720422 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:20.720435 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:20.720502 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:20.720718 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:20.720719 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:20.720851 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:20.720947 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:20.720952 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.721726 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.721729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:20.722257 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:20.722266 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.722678 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.722681 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:20.722782 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:20.723251 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:20.723263 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:20.725009 (+ 1746us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103}
W20260812 19:04:20.725287 4662 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:20.720346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:47961 (local address 127.25.130.62:33683)
0812 19:04:20.720466 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:20.720470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:20.720483 (+ 13us) server_negotiation.cc:407] Connection header received
0812 19:04:20.720579 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:20.720582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:20.720623 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:20.720691 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:20.721060 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.721616 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.722374 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.722579 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.722662 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:20.725179 (+ 2517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.3:47961: BlockingRecv error: recv got EOF from 127.25.130.3:47961 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 19:04:20.726805 4666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:20.718022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44292 (local address 127.25.130.62:33683)
0812 19:04:20.718165 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:20.718178 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:20.718193 (+ 15us) server_negotiation.cc:407] Connection header received
0812 19:04:20.718235 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:20.718238 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:20.718282 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:20.718365 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:20.719257 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.719932 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.721022 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.721259 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.721345 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:20.722769 (+ 1424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:20.722775 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:20.722791 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:20.722821 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:20.724476 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:20.724921 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:20.724927 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:20.724932 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:20.725066 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:20.725464 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:20.725468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:20.725470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:20.725922 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:20.726136 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:20.726438 (+ 302us) server_negotiation.cc:299] Negotiation successful
0812 19:04:20.726693 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Aug 12 19:04:20 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:20 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:20.730881 4970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:20.725842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.3:38527)
0812 19:04:20.726003 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:20.726034 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:20.726109 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:20.726337 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:20.726340 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:20.726475 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:20.726557 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:20.726564 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.727418 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.727421 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:20.728096 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:20.728106 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.728550 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.728552 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:20.728623 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:20.729064 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:20.729078 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:20.730780 (+ 1702us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100}
W20260812 19:04:20.730998 4662 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:20.725957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:38527 (local address 127.25.130.62:33683)
0812 19:04:20.726109 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:20.726113 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:20.726125 (+ 12us) server_negotiation.cc:407] Connection header received
0812 19:04:20.726207 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:20.726210 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:20.726249 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:20.726311 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:20.726697 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.727316 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.728214 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:20.728428 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:20.728522 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:20.730904 (+ 2382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.3:38527: BlockingRecv error: recv got EOF from 127.25.130.3:38527 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
W20260812 19:04:20.731082 4967 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 19:04:21 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:21 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:21.359388 4972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:21.353328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.1:44135)
0812 19:04:21.353698 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:21.353713 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:21.353816 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:21.354109 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:21.354113 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:21.354319 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:21.354486 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:21.354495 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:21.355487 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:21.355489 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:21.356103 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:21.356113 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:21.356560 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:21.356563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:21.356666 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:21.357209 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:21.357222 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:21.359227 (+ 2005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":124,"threads_started":1}
W20260812 19:04:21.359614 4974 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:21.353504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:44135 (local address 127.25.130.62:33683)
0812 19:04:21.353766 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:21.353771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:21.353789 (+ 18us) server_negotiation.cc:407] Connection header received
0812 19:04:21.353921 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:21.353923 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:21.353972 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:21.354051 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:21.354637 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:21.355368 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:21.356205 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:21.356431 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:21.356509 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:21.359372 (+ 2863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.1:44135: BlockingRecv error: recv got EOF from 127.25.130.1:44135 (error 108)
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":65,"threads_started":1}
Aug 12 19:04:21 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:21 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:21.548452 4975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:21.542712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.2:37057)
0812 19:04:21.543034 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:21.543052 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:21.543214 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:21.543479 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:21.543482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:21.543687 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:21.543809 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:21.543817 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:21.544661 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:21.544665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:21.545293 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:21.545305 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:21.545801 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:21.545803 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:21.545869 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:21.546409 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:21.546422 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:21.548312 (+ 1890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":99,"threads_started":1}
W20260812 19:04:21.548617 4974 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:21.542856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:37057 (local address 127.25.130.62:33683)
0812 19:04:21.543038 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:21.543043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:21.543138 (+ 95us) server_negotiation.cc:407] Connection header received
0812 19:04:21.543313 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:21.543317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:21.543368 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:21.543441 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:21.543960 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:21.544540 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:21.545429 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:21.545702 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:21.545791 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:21.548485 (+ 2694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.2:37057: BlockingRecv error: recv got EOF from 127.25.130.2:37057 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 12 19:04:21 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:21 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:21.737618 4976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:21.731846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.3:47615)
0812 19:04:21.732127 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:21.732141 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:21.732251 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:21.732484 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:21.732486 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:21.732684 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:21.732788 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:21.732795 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:21.733719 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:21.733723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:21.734410 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:21.734425 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:21.734969 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:21.734973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:21.735081 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:21.735574 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:21.735585 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:21.737478 (+ 1893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":84,"threads_started":1}
W20260812 19:04:21.737778 4974 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:21.731965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:47615 (local address 127.25.130.62:33683)
0812 19:04:21.732137 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:21.732141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:21.732227 (+ 86us) server_negotiation.cc:407] Connection header received
0812 19:04:21.732325 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:21.732328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:21.732377 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:21.732456 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:21.732890 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:21.733585 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:21.734552 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:21.734837 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:21.734940 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:21.737656 (+ 2716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.3:47615: BlockingRecv error: recv got EOF from 127.25.130.3:47615 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 12 19:04:22 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:22 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:22.365913 4977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:22.360243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.1:45883)
0812 19:04:22.360516 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:22.360530 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:22.360641 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:22.360867 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:22.360868 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:22.361045 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:22.361151 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:22.361157 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:22.361903 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:22.361906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:22.362598 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:22.362611 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:22.363158 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:22.363163 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:22.363249 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:22.363844 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:22.363856 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:22.365735 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":78,"threads_started":1}
W20260812 19:04:22.366024 4978 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:22.360319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:45883 (local address 127.25.130.62:33683)
0812 19:04:22.360548 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:22.360552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:22.360580 (+ 28us) server_negotiation.cc:407] Connection header received
0812 19:04:22.360714 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:22.360717 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:22.360759 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:22.360844 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:22.361260 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:22.361825 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:22.362749 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:22.363029 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:22.363117 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:22.365886 (+ 2769us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.1:45883: BlockingRecv error: recv got EOF from 127.25.130.1:45883 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":69,"threads_started":1}
Aug 12 19:04:22 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:22 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:22.555420 4979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:22.549371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.2:34711)
0812 19:04:22.549721 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:22.549739 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:22.549871 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:22.550129 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:22.550132 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:22.550340 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:22.550445 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:22.550453 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:22.551392 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:22.551395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:22.552145 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:22.552158 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:22.552740 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:22.552743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:22.552833 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:22.553367 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:22.553382 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:22.555259 (+ 1877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":139,"threads_started":1}
W20260812 19:04:22.555527 4978 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:22.549564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:34711 (local address 127.25.130.62:33683)
0812 19:04:22.549724 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:22.549729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:22.549814 (+ 85us) server_negotiation.cc:407] Connection header received
0812 19:04:22.549967 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:22.549971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:22.550019 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:22.550093 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:22.550592 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:22.551271 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:22.552278 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:22.552625 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:22.552743 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:22.555418 (+ 2675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.2:34711: BlockingRecv error: recv got EOF from 127.25.130.2:34711 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 12 19:04:22 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:22 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:22.750604 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:22.738545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.3:35359)
0812 19:04:22.738823 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:22.738839 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:22.738956 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:22.739207 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:22.739210 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:22.739406 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:22.739504 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:22.739510 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:22.740287 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:22.740291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:22.740862 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:22.740870 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:22.741360 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:22.741363 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:22.741485 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:22.748419 (+ 6934us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:22.748435 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:22.750440 (+ 2005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":88,"threads_started":1}
W20260812 19:04:22.750751 4978 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:22.738739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:35359 (local address 127.25.130.62:33683)
0812 19:04:22.738882 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:22.738887 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:22.738902 (+ 15us) server_negotiation.cc:407] Connection header received
0812 19:04:22.739051 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:22.739055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:22.739101 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:22.739174 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:22.739642 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:22.740179 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:22.740961 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:22.741247 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:22.741331 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:22.750620 (+ 9289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.3:35359: BlockingRecv error: recv got EOF from 127.25.130.3:35359 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 12 19:04:23 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:23 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:23.372186 4981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:23.366873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.1:50289)
0812 19:04:23.367177 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:23.367191 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:23.367299 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:23.367518 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:23.367520 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:23.367708 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:23.367823 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:23.367829 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:23.368583 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:23.368586 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:23.369184 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:23.369207 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:23.369663 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:23.369666 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:23.369732 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:23.370212 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:23.370224 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:23.372055 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":97,"threads_started":1}
W20260812 19:04:23.372344 4983 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:23.366920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:50289 (local address 127.25.130.62:33683)
0812 19:04:23.367165 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:23.367169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:23.367261 (+ 92us) server_negotiation.cc:407] Connection header received
0812 19:04:23.367370 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:23.367372 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:23.367418 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:23.367493 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:23.367917 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:23.368449 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:23.369300 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:23.369574 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:23.369654 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:23.372216 (+ 2562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.1:50289: BlockingRecv error: recv got EOF from 127.25.130.1:50289 (error 108)
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":75,"threads_started":1}
Aug 12 19:04:23 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:23 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:23.561868 4984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:23.556299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.2:60537)
0812 19:04:23.556709 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:23.556721 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:23.556833 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:23.557085 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:23.557088 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:23.557297 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:23.557396 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:23.557402 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:23.558236 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:23.558240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:23.558802 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:23.558812 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:23.559299 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:23.559303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:23.559390 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:23.559855 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:23.559867 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:23.561742 (+ 1875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":84,"threads_started":1}
W20260812 19:04:23.562027 4983 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:23.556373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:60537 (local address 127.25.130.62:33683)
0812 19:04:23.556565 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:23.556570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:23.556799 (+ 229us) server_negotiation.cc:407] Connection header received
0812 19:04:23.556924 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:23.556927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:23.556978 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:23.557043 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:23.557542 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:23.558128 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:23.558921 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:23.559185 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:23.559276 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:23.561909 (+ 2633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.2:60537: BlockingRecv error: recv got EOF from 127.25.130.2:60537 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 12 19:04:23 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:23 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:23.757193 4985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:23.751606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.3:40505)
0812 19:04:23.751888 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:23.751903 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:23.751995 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:23.752279 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:23.752282 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:23.752463 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:23.752566 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:23.752572 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:23.753294 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:23.753299 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:23.753928 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:23.753939 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:23.754378 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:23.754381 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:23.754450 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:23.754970 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:23.754983 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:23.757041 (+ 2058us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":88,"threads_started":1}
W20260812 19:04:23.757267 4983 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:23.751757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:40505 (local address 127.25.130.62:33683)
0812 19:04:23.751925 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:23.751929 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:23.751973 (+ 44us) server_negotiation.cc:407] Connection header received
0812 19:04:23.752095 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:23.752098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:23.752152 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:23.752222 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:23.752661 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:23.753189 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:23.754046 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:23.754268 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:23.754350 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:23.757137 (+ 2787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.3:40505: BlockingRecv error: recv got EOF from 127.25.130.3:40505 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 12 19:04:24 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:24 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:24.378263 4986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:24.373067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.1:42319)
0812 19:04:24.373362 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:24.373376 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:24.373503 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:24.373728 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:24.373730 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:24.373911 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:24.374012 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:24.374017 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:24.374713 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:24.374715 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:24.375276 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:24.375282 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:24.375666 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:24.375668 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:24.375731 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:24.376307 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:24.376320 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:24.378134 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":74,"threads_started":1}
W20260812 19:04:24.378427 4987 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:24.373118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:42319 (local address 127.25.130.62:33683)
0812 19:04:24.373393 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:24.373396 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:24.373430 (+ 34us) server_negotiation.cc:407] Connection header received
0812 19:04:24.373580 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:24.373582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:24.373631 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:24.373701 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:24.374095 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:24.374633 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:24.375354 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:24.375574 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:24.375651 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:24.378295 (+ 2644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.1:42319: BlockingRecv error: recv got EOF from 127.25.130.1:42319 (error 108)
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":97,"threads_started":1}
Aug 12 19:04:24 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:24 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:24.567890 4988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:24.562607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.2:54539)
0812 19:04:24.562921 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:24.562934 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:24.563044 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:24.563289 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:24.563292 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:24.563501 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:24.563597 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:24.563603 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:24.564354 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:24.564358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:24.564913 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:24.564921 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:24.565368 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:24.565372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:24.565495 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:24.565971 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:24.565983 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:24.567720 (+ 1737us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":94,"threads_started":1}
W20260812 19:04:24.567991 4987 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:24.562684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:54539 (local address 127.25.130.62:33683)
0812 19:04:24.562840 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:24.562844 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:24.563002 (+ 158us) server_negotiation.cc:407] Connection header received
0812 19:04:24.563133 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:24.563136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:24.563183 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:24.563263 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:24.563710 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:24.564236 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:24.565030 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:24.565259 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:24.565349 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:24.567884 (+ 2535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.2:54539: BlockingRecv error: recv got EOF from 127.25.130.2:54539 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 19:04:24 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:24 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:24.763823 4990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:24.758180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.3:38257)
0812 19:04:24.758465 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:24.758480 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:24.758596 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:24.758863 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:24.758865 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:24.759069 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:24.759192 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:24.759198 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:24.759946 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:24.759948 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:24.760504 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:24.760512 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:24.760908 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:24.760910 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:24.760976 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:24.761723 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:24.761736 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:24.763700 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":84,"threads_started":1}
W20260812 19:04:24.764010 4987 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:24.758266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:38257 (local address 127.25.130.62:33683)
0812 19:04:24.758421 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:24.758426 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:24.758553 (+ 127us) server_negotiation.cc:407] Connection header received
0812 19:04:24.758688 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:24.758691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:24.758741 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:24.758804 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:24.759307 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:24.759844 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:24.760599 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:24.760824 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:24.760898 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:24.763884 (+ 2986us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.3:38257: BlockingRecv error: recv got EOF from 127.25.130.3:38257 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 12 19:04:25 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:25 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:25.384986 4991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:25.379124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.1:37183)
0812 19:04:25.379517 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:25.379533 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:25.379655 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:25.380053 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:25.380057 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:25.380301 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:25.380433 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:25.380442 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:25.381263 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:25.381266 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:25.382000 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:25.382011 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:25.382455 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:25.382459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:25.382543 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:25.383080 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:25.383106 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:25.384824 (+ 1718us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":97,"threads_started":1}
W20260812 19:04:25.385123 4993 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:25.379268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:37183 (local address 127.25.130.62:33683)
0812 19:04:25.379543 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:25.379547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:25.379608 (+ 61us) server_negotiation.cc:407] Connection header received
0812 19:04:25.379851 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:25.379853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:25.379909 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:25.380003 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:25.380531 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:25.381153 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:25.382113 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:25.382348 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:25.382437 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:25.385002 (+ 2565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.1:37183: BlockingRecv error: recv got EOF from 127.25.130.1:37183 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":62,"threads_started":1}
Aug 12 19:04:25 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:25 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:25.574604 4994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:25.568887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.2:41629)
0812 19:04:25.569239 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:25.569254 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:25.569350 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:25.569613 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:25.569615 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:25.569807 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:25.569907 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:25.569913 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:25.570844 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:25.570847 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:25.571553 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:25.571566 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:25.572063 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:25.572065 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:25.572137 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:25.572605 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:25.572618 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:25.574465 (+ 1847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":85,"threads_started":1}
W20260812 19:04:25.574733 4993 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:25.569147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:41629 (local address 127.25.130.62:33683)
0812 19:04:25.569308 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:25.569312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:25.569328 (+ 16us) server_negotiation.cc:407] Connection header received
0812 19:04:25.569437 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:25.569439 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:25.569505 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:25.569576 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:25.570031 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:25.570718 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:25.571700 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:25.571961 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:25.572048 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:25.574620 (+ 2572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.2:41629: BlockingRecv error: recv got EOF from 127.25.130.2:41629 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 12 19:04:25 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:25 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:25.769948 4995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:25.764615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.3:60629)
0812 19:04:25.764889 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:25.764903 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:25.765001 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:25.765223 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:25.765225 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:25.765413 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:25.765545 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:25.765550 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:25.766321 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:25.766325 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:25.766895 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:25.766905 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:25.767352 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:25.767355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:25.767422 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:25.767921 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:25.767934 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:25.769819 (+ 1885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":81,"threads_started":1}
W20260812 19:04:25.770092 4993 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:25.764678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:60629 (local address 127.25.130.62:33683)
0812 19:04:25.764820 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:25.764824 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:25.764967 (+ 143us) server_negotiation.cc:407] Connection header received
0812 19:04:25.765083 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:25.765085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:25.765130 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:25.765195 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:25.765637 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:25.766194 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:25.767006 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:25.767243 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:25.767319 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:25.769977 (+ 2658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.3:60629: BlockingRecv error: recv got EOF from 127.25.130.3:60629 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 12 19:04:26 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:26 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:26.391564 4996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:26.385925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.1:56557)
0812 19:04:26.386235 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:26.386249 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:26.386352 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:26.386595 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:26.386597 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:26.386785 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:26.386890 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:26.386895 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:26.387628 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:26.387631 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:26.388214 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:26.388223 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:26.388622 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:26.388626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:26.388709 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:26.389283 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:26.389299 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:26.391430 (+ 2131us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":104,"threads_started":1}
W20260812 19:04:26.391713 4997 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:26.386031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:56557 (local address 127.25.130.62:33683)
0812 19:04:26.386296 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:26.386300 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:26.386315 (+ 15us) server_negotiation.cc:407] Connection header received
0812 19:04:26.386443 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:26.386445 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:26.386491 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:26.386569 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:26.386983 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:26.387531 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:26.388301 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:26.388517 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:26.388584 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:26.391588 (+ 3004us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.1:56557: BlockingRecv error: recv got EOF from 127.25.130.1:56557 (error 108)
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":83,"threads_started":1}
Aug 12 19:04:26 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:26 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:26.581067 4998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:26.575484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.2:42885)
0812 19:04:26.575783 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:26.575796 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:26.575906 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:26.576174 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:26.576176 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:26.576413 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:26.576521 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:26.576528 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:26.577321 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:26.577325 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:26.578069 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:26.578081 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:26.578569 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:26.578573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:26.578659 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:26.579125 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:26.579137 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:26.580922 (+ 1785us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":110,"threads_started":1}
W20260812 19:04:26.581220 4997 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:26.575574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:42885 (local address 127.25.130.62:33683)
0812 19:04:26.575764 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:26.575769 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:26.575865 (+ 96us) server_negotiation.cc:407] Connection header received
0812 19:04:26.575994 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:26.575996 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:26.576087 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:26.576149 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:26.576638 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:26.577207 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:26.578177 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:26.578436 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:26.578536 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:26.581080 (+ 2544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.2:42885: BlockingRecv error: recv got EOF from 127.25.130.2:42885 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 12 19:04:26 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:26 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:26.776379 4999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:26.770905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.3:44891)
0812 19:04:26.771222 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:26.771238 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:26.771362 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:26.771629 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:26.771632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:26.771857 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:26.771959 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:26.771965 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:26.772758 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:26.772762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:26.773324 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:26.773334 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:26.773849 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:26.773853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:26.773941 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:26.774450 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:26.774462 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:26.776214 (+ 1752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":132,"threads_started":1}
W20260812 19:04:26.776537 4997 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:26.771018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:44891 (local address 127.25.130.62:33683)
0812 19:04:26.771192 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:26.771204 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:26.771313 (+ 109us) server_negotiation.cc:407] Connection header received
0812 19:04:26.771457 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:26.771460 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:26.771512 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:26.771585 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:26.772084 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:26.772636 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:26.773478 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:26.773728 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:26.773825 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:26.776418 (+ 2593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.3:44891: BlockingRecv error: recv got EOF from 127.25.130.3:44891 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 12 19:04:27 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:27 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:27.397780 5001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:27.392361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.1:42415)
0812 19:04:27.392707 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:27.392721 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:27.392825 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:27.393048 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:27.393050 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:27.393267 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:27.393372 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:27.393378 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:27.394190 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:27.394193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:27.394774 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:27.394784 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:27.395203 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:27.395206 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:27.395288 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:27.395828 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:27.395841 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:27.397650 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":75,"threads_started":1}
W20260812 19:04:27.397965 5002 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:27.392435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:42415 (local address 127.25.130.62:33683)
0812 19:04:27.392709 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:27.392713 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:27.392783 (+ 70us) server_negotiation.cc:407] Connection header received
0812 19:04:27.392897 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:27.392900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:27.392946 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:27.393020 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:27.393479 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:27.394047 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:27.394874 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:27.395094 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:27.395163 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:27.397814 (+ 2651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.1:42415: BlockingRecv error: recv got EOF from 127.25.130.1:42415 (error 108)
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":87,"threads_started":1}
Aug 12 19:04:27 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:27 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:27.587637 5004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:27.581974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.2:59491)
0812 19:04:27.582371 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:27.582386 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:27.582504 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:27.582769 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:27.582772 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:27.582983 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:27.583099 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:27.583105 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:27.583941 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:27.583945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:27.584515 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:27.584524 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:27.584994 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:27.584998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:27.585083 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:27.585615 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:27.585627 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:27.587463 (+ 1836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":99,"threads_started":1}
W20260812 19:04:27.587735 5002 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:27.582076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:59491 (local address 127.25.130.62:33683)
0812 19:04:27.582282 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:27.582287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:27.582460 (+ 173us) server_negotiation.cc:407] Connection header received
0812 19:04:27.582611 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:27.582614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:27.582665 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:27.582731 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:27.583259 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:27.583830 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:27.584642 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:27.584883 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:27.584974 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:27.587626 (+ 2652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.2:59491: BlockingRecv error: recv got EOF from 127.25.130.2:59491 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 12 19:04:27 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:27 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:27.788004 5005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:27.777308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.3:60371)
0812 19:04:27.777622 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:27.777636 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:27.777745 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:27.777966 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:27.777967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:27.778163 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:27.778265 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:27.778272 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:27.779075 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:27.779077 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:27.779652 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:27.779661 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:27.780044 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:27.780047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:27.780105 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:27.785829 (+ 5724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:27.785844 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:27.787867 (+ 2023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":98,"threads_started":1}
W20260812 19:04:27.788225 5002 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:27.777415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:60371 (local address 127.25.130.62:33683)
0812 19:04:27.777577 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:27.777581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:27.777704 (+ 123us) server_negotiation.cc:407] Connection header received
0812 19:04:27.777818 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:27.777821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:27.777869 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:27.777940 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:27.778412 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:27.778968 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:27.779745 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:27.779960 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:27.780030 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:27.788081 (+ 8051us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.3:60371: BlockingRecv error: recv got EOF from 127.25.130.3:60371 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 12 19:04:28 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:28 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:28.404031 5006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:28.398613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.1:52927)
0812 19:04:28.398888 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:28.398901 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:28.398993 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:28.399266 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:28.399268 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:28.399454 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:28.399563 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:28.399569 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:28.400317 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:28.400321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:28.400894 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:28.400903 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:28.401371 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:28.401375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:28.401502 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:28.402058 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:28.402070 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:28.403889 (+ 1819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":99,"threads_started":1}
W20260812 19:04:28.404187 5007 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:28.398807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:52927 (local address 127.25.130.62:33683)
0812 19:04:28.399028 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:28.399032 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:28.399067 (+ 35us) server_negotiation.cc:407] Connection header received
0812 19:04:28.399104 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:28.399106 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:28.399149 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:28.399219 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:28.399654 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:28.400194 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:28.401002 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:28.401242 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:28.401320 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:28.404041 (+ 2721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.1:52927: BlockingRecv error: recv got EOF from 127.25.130.1:52927 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":62,"threads_started":1}
Aug 12 19:04:28 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:28 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:28.594023 5008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:28.588470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.2:48947)
0812 19:04:28.588761 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:28.588775 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:28.588877 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:28.589102 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:28.589104 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:28.589292 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:28.589396 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:28.589401 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:28.590385 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:28.590388 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:28.591001 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:28.591013 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:28.591456 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:28.591459 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:28.591521 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:28.592021 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:28.592033 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:28.593887 (+ 1854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":101,"threads_started":1}
W20260812 19:04:28.594189 5007 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:28.588582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:48947 (local address 127.25.130.62:33683)
0812 19:04:28.588721 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:28.588725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:28.588837 (+ 112us) server_negotiation.cc:407] Connection header received
0812 19:04:28.588947 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:28.588949 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:28.588994 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:28.589075 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:28.589541 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:28.590268 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:28.591121 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:28.591357 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:28.591441 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:28.594061 (+ 2620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.2:48947: BlockingRecv error: recv got EOF from 127.25.130.2:48947 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 12 19:04:28 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:28 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:28.794498 5009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:28.788880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.3:41111)
0812 19:04:28.789313 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:28.789327 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:28.789441 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:28.789752 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:28.789756 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:28.790055 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:28.790180 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:28.790187 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:28.790928 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:28.790930 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:28.791569 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:28.791580 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:28.791982 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:28.791985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:28.792071 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:28.792577 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:28.792591 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:28.794374 (+ 1783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":81,"threads_started":1}
W20260812 19:04:28.794636 5007 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:28.788995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:41111 (local address 127.25.130.62:33683)
0812 19:04:28.789137 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:28.789143 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:28.789402 (+ 259us) server_negotiation.cc:407] Connection header received
0812 19:04:28.789557 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:28.789560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:28.789621 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:28.789701 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:28.790289 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:28.790827 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:28.791675 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:28.791902 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:28.791972 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:28.794519 (+ 2547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.3:41111: BlockingRecv error: recv got EOF from 127.25.130.3:41111 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 12 19:04:29 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:29 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:29.410595 5010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:29.404905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.1:59685)
0812 19:04:29.405327 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:29.405343 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:29.405484 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:29.405788 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:29.405791 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:29.406042 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:29.406175 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:29.406182 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:29.406950 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:29.406953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:29.407533 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:29.407542 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:29.407926 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:29.407929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:29.407988 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:29.408470 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:29.408481 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:29.410419 (+ 1938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":94,"threads_started":1}
W20260812 19:04:29.410720 5011 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:29.404973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:59685 (local address 127.25.130.62:33683)
0812 19:04:29.405254 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:29.405259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:29.405417 (+ 158us) server_negotiation.cc:407] Connection header received
0812 19:04:29.405575 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:29.405578 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:29.405646 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:29.405741 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:29.406270 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:29.406815 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:29.407628 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:29.407845 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:29.407913 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:29.410596 (+ 2683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.1:59685: BlockingRecv error: recv got EOF from 127.25.130.1:59685 (error 108)
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":81,"threads_started":1}
Aug 12 19:04:29 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:29 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:29.600636 5012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:29.594892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.2:57083)
0812 19:04:29.595267 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:29.595284 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:29.595431 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:29.595734 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:29.595738 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:29.595963 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:29.596081 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:29.596090 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:29.596980 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:29.596983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:29.597577 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:29.597587 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:29.598012 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:29.598014 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:29.598075 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:29.598555 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:29.598567 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:29.600483 (+ 1916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":124,"threads_started":1}
W20260812 19:04:29.600780 5011 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:29.595051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:57083 (local address 127.25.130.62:33683)
0812 19:04:29.595252 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:29.595257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:29.595362 (+ 105us) server_negotiation.cc:407] Connection header received
0812 19:04:29.595522 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:29.595525 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:29.595582 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:29.595670 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:29.596217 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:29.596873 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:29.597693 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:29.597918 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:29.598001 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:29.600664 (+ 2663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.2:57083: BlockingRecv error: recv got EOF from 127.25.130.2:57083 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 12 19:04:29 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:29 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:29.801052 5013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:29.795285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.3:49483)
0812 19:04:29.795574 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:29.795588 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:29.795692 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:29.795922 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:29.795924 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:29.796103 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:29.796211 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:29.796216 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:29.797207 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:29.797211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:29.797939 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:29.797950 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:29.798364 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:29.798366 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:29.798426 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:29.798889 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:29.798900 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:29.800845 (+ 1945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":99,"threads_started":1}
W20260812 19:04:29.801101 5011 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:29.795375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:49483 (local address 127.25.130.62:33683)
0812 19:04:29.795498 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:29.795501 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:29.795651 (+ 150us) server_negotiation.cc:407] Connection header received
0812 19:04:29.795768 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:29.795770 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:29.795818 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:29.795896 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:29.796328 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:29.797044 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:29.798052 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:29.798274 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:29.798355 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:29.800998 (+ 2643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.3:49483: BlockingRecv error: recv got EOF from 127.25.130.3:49483 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 19:04:30 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:30 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:30.418308 5015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:30.411784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.1:53911)
0812 19:04:30.412151 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:30.412168 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:30.412285 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:30.412560 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:30.412562 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:30.412798 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:30.412948 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:30.412956 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:30.413874 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:30.413877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:30.414632 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:30.414648 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:30.415248 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:30.415252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:30.415350 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:30.415968 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:30.415981 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:30.418141 (+ 2160us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":109,"threads_started":1}
W20260812 19:04:30.418485 5016 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:30.411976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:53911 (local address 127.25.130.62:33683)
0812 19:04:30.412283 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:30.412289 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:30.412310 (+ 21us) server_negotiation.cc:407] Connection header received
0812 19:04:30.412369 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:30.412372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:30.412426 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:30.412536 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:30.413062 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:30.413711 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:30.414783 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:30.415117 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:30.415214 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:30.418351 (+ 3137us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.1:53911: BlockingRecv error: recv got EOF from 127.25.130.1:53911 (error 108)
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":92,"threads_started":1}
Aug 12 19:04:30 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:30 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:30.608467 5017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:30.601872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.2:57695)
0812 19:04:30.602226 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:30.602241 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:30.602374 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:30.602629 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:30.602631 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:30.602876 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:30.603004 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:30.603017 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:30.603837 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:30.603840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:30.604506 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:30.604521 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:30.605145 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:30.605149 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:30.605229 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:30.605986 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:30.606002 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:30.608303 (+ 2301us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":101,"threads_started":1}
W20260812 19:04:30.608696 5016 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:30.601974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:57695 (local address 127.25.130.62:33683)
0812 19:04:30.602199 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:30.602204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:30.602323 (+ 119us) server_negotiation.cc:407] Connection header received
0812 19:04:30.602462 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:30.602465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:30.602519 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:30.602594 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:30.603109 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:30.603736 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:30.604667 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:30.605002 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:30.605107 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:30.608562 (+ 3455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.2:57695: BlockingRecv error: recv got EOF from 127.25.130.2:57695 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 12 19:04:30 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:30 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:30.808589 5018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:30.802180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.3:38325)
0812 19:04:30.802629 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:30.802649 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:30.802812 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:30.803125 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:30.803128 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:30.803380 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:30.803514 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:30.803522 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:30.804375 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:30.804378 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:30.804976 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:30.804986 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:30.805519 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:30.805523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:30.805622 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:30.806258 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:30.806272 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:30.808425 (+ 2153us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":187,"threads_started":1}
W20260812 19:04:30.808804 5016 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:30.802484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:38325 (local address 127.25.130.62:33683)
0812 19:04:30.802667 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:30.802673 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:30.802725 (+ 52us) server_negotiation.cc:407] Connection header received
0812 19:04:30.802927 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:30.802930 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:30.802990 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:30.803064 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:30.803653 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:30.804250 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:30.805103 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:30.805356 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:30.805485 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:30.808659 (+ 3174us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.3:38325: BlockingRecv error: recv got EOF from 127.25.130.3:38325 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 12 19:04:31 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:31 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:31.425204 5019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:31.419365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.1:56189)
0812 19:04:31.419703 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:31.419716 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:31.419822 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:31.420071 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:31.420073 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:31.420283 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:31.420405 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:31.420411 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:31.421151 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:31.421153 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:31.421772 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:31.421780 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:31.422301 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:31.422305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:31.422390 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:31.423051 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:31.423068 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:31.425052 (+ 1984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":106,"threads_started":1}
W20260812 19:04:31.425348 5021 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:31.419470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:56189 (local address 127.25.130.62:33683)
0812 19:04:31.419754 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:31.419758 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:31.419774 (+ 16us) server_negotiation.cc:407] Connection header received
0812 19:04:31.419911 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:31.419914 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:31.419963 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:31.420041 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:31.420501 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:31.421068 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:31.421884 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:31.422169 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:31.422265 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:31.425202 (+ 2937us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.1:56189: BlockingRecv error: recv got EOF from 127.25.130.1:56189 (error 108)
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":74,"threads_started":1}
Aug 12 19:04:31 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:31 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:31.615257 5022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:31.609493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.2:55187)
0812 19:04:31.609823 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:31.609839 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:31.609978 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:31.610259 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:31.610262 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:31.610476 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:31.610600 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:31.610607 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:31.611398 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:31.611401 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:31.612086 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:31.612100 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:31.612531 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:31.612535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:31.612627 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:31.613159 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:31.613171 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:31.615128 (+ 1957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":101,"threads_started":1}
W20260812 19:04:31.615439 5021 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:31.609624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:55187 (local address 127.25.130.62:33683)
0812 19:04:31.609820 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:31.609824 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:31.609922 (+ 98us) server_negotiation.cc:407] Connection header received
0812 19:04:31.610084 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:31.610088 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:31.610141 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:31.610209 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:31.610732 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:31.611282 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:31.612181 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:31.612424 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:31.612512 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:31.615316 (+ 2804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.2:55187: BlockingRecv error: recv got EOF from 127.25.130.2:55187 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 12 19:04:31 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:31 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:31.815616 5023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:31.809710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.3:41835)
0812 19:04:31.810018 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:31.810034 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:31.810151 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:31.810387 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:31.810389 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:31.810591 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:31.810706 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:31.810713 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:31.811557 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:31.811561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:31.812319 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:31.812334 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:31.812839 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:31.812843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:31.812915 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:31.813477 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:31.813491 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:31.815439 (+ 1948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":84,"threads_started":1}
W20260812 19:04:31.815734 5021 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:31.809830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:41835 (local address 127.25.130.62:33683)
0812 19:04:31.809968 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:31.809971 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:31.810092 (+ 121us) server_negotiation.cc:407] Connection header received
0812 19:04:31.810244 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:31.810246 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:31.810294 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:31.810355 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:31.810826 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:31.811419 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:31.812459 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:31.812697 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:31.812791 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:31.815610 (+ 2819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.3:41835: BlockingRecv error: recv got EOF from 127.25.130.3:41835 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 12 19:04:32 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:32 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:32.432587 5024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:32.426183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.1:51349)
0812 19:04:32.426524 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:32.426540 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:32.426655 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:32.426959 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:32.426962 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:32.427230 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:32.427403 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:32.427411 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:32.428195 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:32.428197 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:32.428827 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:32.428836 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:32.429354 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:32.429358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:32.429438 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:32.430104 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:32.430119 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:32.432413 (+ 2294us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":102,"threads_started":1}
W20260812 19:04:32.432747 5026 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:32.426230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:51349 (local address 127.25.130.62:33683)
0812 19:04:32.426567 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:32.426570 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:32.426589 (+ 19us) server_negotiation.cc:407] Connection header received
0812 19:04:32.426752 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:32.426755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:32.426811 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:32.426900 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:32.427509 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:32.428108 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:32.428947 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:32.429213 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:32.429298 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:32.432602 (+ 3304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.1:51349: BlockingRecv error: recv got EOF from 127.25.130.1:51349 (error 108)
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":136,"threads_started":1}
Aug 12 19:04:32 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:32 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:32.622591 5027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:32.616336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.2:44031)
0812 19:04:32.616723 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:32.616740 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:32.616885 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:32.617158 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:32.617161 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:32.617425 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:32.617594 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:32.617602 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:32.618438 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:32.618442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:32.619040 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:32.619048 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:32.619522 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:32.619526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:32.619617 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:32.620206 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:32.620218 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:32.622399 (+ 2181us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":153,"threads_started":1}
W20260812 19:04:32.622709 5026 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:32.616438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:44031 (local address 127.25.130.62:33683)
0812 19:04:32.616609 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:32.616614 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:32.616832 (+ 218us) server_negotiation.cc:407] Connection header received
0812 19:04:32.616983 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:32.616986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:32.617042 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:32.617113 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:32.617715 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:32.618323 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:32.619158 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:32.619410 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:32.619502 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:32.622586 (+ 3084us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.2:44031: BlockingRecv error: recv got EOF from 127.25.130.2:44031 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 12 19:04:32 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:32 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:32.828315 5028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:32.816639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.3:48081)
0812 19:04:32.817038 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:32.817052 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:32.817178 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:32.817469 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:32.817472 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:32.817709 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:32.817858 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:32.817865 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:32.818650 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:32.818653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:32.819268 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:32.819280 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:32.819831 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:32.819835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:32.819917 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:32.825747 (+ 5830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:32.825765 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:32.828144 (+ 2379us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":119,"threads_started":1}
W20260812 19:04:32.828637 5026 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:32.816754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:48081 (local address 127.25.130.62:33683)
0812 19:04:32.816959 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:32.816963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:32.817124 (+ 161us) server_negotiation.cc:407] Connection header received
0812 19:04:32.817262 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:32.817264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:32.817325 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:32.817403 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:32.817960 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:32.818543 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:32.819413 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:32.819681 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:32.819799 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:32.828482 (+ 8683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.3:48081: BlockingRecv error: recv got EOF from 127.25.130.3:48081 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 12 19:04:33 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:33 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:33.440042 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:33.433856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.1:34453)
0812 19:04:33.434229 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:33.434244 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:33.434371 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:33.434617 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:33.434619 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:33.434845 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:33.434969 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:33.434976 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:33.435839 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:33.435842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:33.436558 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:33.436575 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:33.437156 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:33.437160 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:33.437234 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:33.437873 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:33.437891 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:33.439871 (+ 1980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":109,"threads_started":1}
W20260812 19:04:33.440179 5030 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:33.433974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:34453 (local address 127.25.130.62:33683)
0812 19:04:33.434250 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:33.434254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:33.434323 (+ 69us) server_negotiation.cc:407] Connection header received
0812 19:04:33.434452 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:33.434454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:33.434504 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:33.434588 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:33.435112 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:33.435696 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:33.436696 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:33.437002 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:33.437149 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:33.440031 (+ 2882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.1:34453: BlockingRecv error: recv got EOF from 127.25.130.1:34453 (error 108)
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":81,"threads_started":1}
Aug 12 19:04:33 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:33 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:33.629423 5031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:33.623683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.2:55847)
0812 19:04:33.624040 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:33.624055 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:33.624184 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:33.624401 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:33.624403 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:33.624609 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:33.624723 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:33.624728 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:33.625719 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:33.625723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:33.626342 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:33.626352 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:33.626815 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:33.626818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:33.626905 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:33.627436 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:33.627448 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:33.629249 (+ 1801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":101,"threads_started":1}
W20260812 19:04:33.629716 5030 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:33.623910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:55847 (local address 127.25.130.62:33683)
0812 19:04:33.624046 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:33.624050 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:33.624124 (+ 74us) server_negotiation.cc:407] Connection header received
0812 19:04:33.624256 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:33.624258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:33.624304 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:33.624370 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:33.624846 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:33.625596 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:33.626463 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:33.626701 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:33.626790 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:33.629404 (+ 2614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.2:55847: BlockingRecv error: recv got EOF from 127.25.130.2:55847 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 12 19:04:33 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:33 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:33.834945 5032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:33.829205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.3:47067)
0812 19:04:33.829575 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:33.829589 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:33.829696 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:33.829945 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:33.829947 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:33.830166 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:33.830287 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:33.830293 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:33.831099 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:33.831103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:33.831678 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:33.831687 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:33.832153 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:33.832156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:33.832230 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:33.832765 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:33.832776 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:33.834806 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":124,"threads_started":1}
W20260812 19:04:33.835108 5030 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:33.829330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:47067 (local address 127.25.130.62:33683)
0812 19:04:33.829537 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:33.829541 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:33.829654 (+ 113us) server_negotiation.cc:407] Connection header received
0812 19:04:33.829771 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:33.829773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:33.829821 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:33.829897 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:33.830402 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:33.830936 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:33.831789 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:33.832016 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:33.832098 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:33.834973 (+ 2875us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.3:47067: BlockingRecv error: recv got EOF from 127.25.130.3:47067 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 12 19:04:34 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:34 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:34.447067 5033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:34.441065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.1:38011)
0812 19:04:34.441375 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:34.441390 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:34.441544 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:34.441866 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:34.441869 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:34.442074 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:34.442188 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:34.442194 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:34.443153 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:34.443157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:34.443902 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:34.443916 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:34.444408 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:34.444411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:34.444472 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:34.444967 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:34.444979 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:34.446915 (+ 1936us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":92,"threads_started":1}
W20260812 19:04:34.447207 5034 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:34.441234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:38011 (local address 127.25.130.62:33683)
0812 19:04:34.441554 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:34.441559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:34.441575 (+ 16us) server_negotiation.cc:407] Connection header received
0812 19:04:34.441646 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:34.441648 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:34.441694 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:34.441771 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:34.442336 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:34.443018 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:34.444039 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:34.444303 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:34.444399 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:34.447083 (+ 2684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.1:38011: BlockingRecv error: recv got EOF from 127.25.130.1:38011 (error 108)
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":67,"threads_started":1}
Aug 12 19:04:34 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:34 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:34.636056 5035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:34.630325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.2:52927)
0812 19:04:34.630594 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:34.630608 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:34.630716 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:34.630952 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:34.630954 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:34.631139 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:34.631248 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:34.631253 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:34.632037 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:34.632040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:34.632732 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:34.632746 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:34.633301 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:34.633305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:34.633394 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:34.633953 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:34.633967 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:34.635885 (+ 1918us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":79,"threads_started":1}
W20260812 19:04:34.636159 5034 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:34.630422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:52927 (local address 127.25.130.62:33683)
0812 19:04:34.630602 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:34.630605 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:34.630740 (+ 135us) server_negotiation.cc:407] Connection header received
0812 19:04:34.630792 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:34.630795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:34.630847 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:34.630926 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:34.631361 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:34.631910 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:34.632867 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:34.633172 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:34.633276 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:34.636055 (+ 2779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.2:52927: BlockingRecv error: recv got EOF from 127.25.130.2:52927 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 12 19:04:34 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:34 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.3@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:34.841578 5036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:34.835843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.3:47291)
0812 19:04:34.836167 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:34.836182 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:34.836283 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:34.836552 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:34.836555 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:34.836760 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:34.836864 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:34.836870 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:34.837662 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:34.837665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:34.838248 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:34.838256 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:34.838680 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:34.838683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:34.838746 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:34.839229 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:34.839241 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:34.841371 (+ 2130us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":115,"threads_started":1}
W20260812 19:04:34.841737 5034 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:34.835950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:47291 (local address 127.25.130.62:33683)
0812 19:04:34.836096 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:34.836100 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:34.836244 (+ 144us) server_negotiation.cc:407] Connection header received
0812 19:04:34.836358 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:34.836360 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:34.836409 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:34.836466 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:34.836983 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:34.837566 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:34.838351 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:34.838592 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:34.838671 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:34.841608 (+ 2937us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.3:47291: BlockingRecv error: recv got EOF from 127.25.130.3:47291 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 12 19:04:35 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:35 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.1@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:35.454531 5037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:35.448262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.1:59539)
0812 19:04:35.448659 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:35.448676 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:35.448790 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:35.449067 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:35.449069 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:35.449304 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:35.449435 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:35.449476 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:35.450383 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:35.450387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:35.450993 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:35.451003 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:35.451498 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:35.451502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:35.451596 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:35.452126 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:35.452138 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:35.454296 (+ 2158us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":127,"threads_started":1}
W20260812 19:04:35.454607 5038 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:35.448412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:59539 (local address 127.25.130.62:33683)
0812 19:04:35.448731 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:35.448735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:35.448750 (+ 15us) server_negotiation.cc:407] Connection header received
0812 19:04:35.448896 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:35.448899 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:35.448948 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:35.449033 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:35.449562 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:35.450253 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:35.451121 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:35.451377 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:35.451467 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:35.454470 (+ 3003us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.1:59539: BlockingRecv error: recv got EOF from 127.25.130.1:59539 (error 108)
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":97,"threads_started":1}
Aug 12 19:04:35 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 19:04:35 dist-test-slave-49h3 krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.130.2@KRBTEST.COM for oryx/127.25.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 19:04:35.643710 5039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:35.637169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33683 (local address 127.25.130.2:58715)
0812 19:04:35.637575 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:35.637593 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:35.637724 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:35.638013 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:35.638016 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:35.638266 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:35.638424 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:35.638432 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:35.639265 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:35.639269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:35.639924 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:35.639935 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:35.640497 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:35.640502 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:35.640624 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:35.641301 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:35.641315 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:35.643533 (+ 2218us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.130.62:33683: Server oryx/127.25.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":155,"threads_started":1}
W20260812 19:04:35.643816 5038 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:35.637339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:58715 (local address 127.25.130.62:33683)
0812 19:04:35.637566 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:35.637571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:35.637668 (+ 97us) server_negotiation.cc:407] Connection header received
0812 19:04:35.637837 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:35.637840 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:35.637896 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:35.637969 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:35.638546 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:35.639154 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:35.640066 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:35.640321 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:35.640434 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:35.643702 (+ 3268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.130.2:58715: BlockingRecv error: recv got EOF from 127.25.130.2:58715 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 19:04:35.723016 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 4484
I20260812 19:04:35.734860 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 4665
I20260812 19:04:35.745625 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 4817
I20260812 19:04:35.757179 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 4401
2026-08-12T19:04:35Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18109 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T19:04:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:04:35Z Disabled control of system clock
I20260812 19:04:35.798432 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:46115
--webserver_interface=127.25.130.62
--webserver_port=0
--builtin_ntp_servers=127.25.130.20:43329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:46115
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 19:04:35.945627 5043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:35.945958 5043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:35.945998 5043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:35.950114 5043 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:04:35.950181 5043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:35.950204 5043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:35.950230 5043 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:04:35.950261 5043 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:04:35.955174 5043 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:43329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:46115
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:46115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:35.956511 5043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:35.958314 5043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:35.968237 5049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:35.968218 5052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:35.968240 5050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:35.968616 5043 server_base.cc:1034] running on GCE node
I20260812 19:04:35.969386 5043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:35.970765 5043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:35.971935 5043 hybrid_clock.cc:648] HybridClock initialized: now 1786561475971913 us; error 42 us; skew 500 ppm
I20260812 19:04:35.974188 5043 webserver.cc:460] Webserver started at http://127.25.130.62:44155/ using document root <none> and password file <none>
I20260812 19:04:35.974923 5043 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:35.974998 5043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:35.975260 5043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:35.977869 5043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "7096045f3b9341de9685ac2b82766fd4"
format_stamp: "Formatted at 2026-08-12 19:04:35 on dist-test-slave-49h3"
server_key: "5a20303ff44a750819db0dce90ae68c5"
server_key_iv: "bb9382a9f4c1959b5809dc3a12e2ec8f"
server_key_version: "encryptionkey@0"
I20260812 19:04:35.978543 5043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "7096045f3b9341de9685ac2b82766fd4"
format_stamp: "Formatted at 2026-08-12 19:04:35 on dist-test-slave-49h3"
server_key: "5a20303ff44a750819db0dce90ae68c5"
server_key_iv: "bb9382a9f4c1959b5809dc3a12e2ec8f"
server_key_version: "encryptionkey@0"
I20260812 19:04:35.982868 5043 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:04:35.985700 5058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:35.987293 5043 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:04:35.987440 5043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "7096045f3b9341de9685ac2b82766fd4"
format_stamp: "Formatted at 2026-08-12 19:04:35 on dist-test-slave-49h3"
server_key: "5a20303ff44a750819db0dce90ae68c5"
server_key_iv: "bb9382a9f4c1959b5809dc3a12e2ec8f"
server_key_version: "encryptionkey@0"
I20260812 19:04:35.987612 5043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:36.007669 5043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:36.008591 5043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:36.008782 5043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:36.017356 5043 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:46115
I20260812 19:04:36.017374 5122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:46115 every 8 connection(s)
I20260812 19:04:36.018615 5043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:04:36.022739 5123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:36.025640 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 5043
I20260812 19:04:36.025767 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:04:36.026105 26120 external_mini_cluster.cc:1428] Setting key 700a1a15de605f2233f127e4ba8442ef
I20260812 19:04:36.030530 5123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7096045f3b9341de9685ac2b82766fd4: Bootstrap starting.
I20260812 19:04:36.033900 5123 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7096045f3b9341de9685ac2b82766fd4: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:36.035023 5123 log.cc:824] T 00000000000000000000000000000000 P 7096045f3b9341de9685ac2b82766fd4: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:36.037902 5123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7096045f3b9341de9685ac2b82766fd4: No bootstrap required, opened a new log
I20260812 19:04:36.038609 5126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:36.027896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49582 (local address 127.25.130.62:46115)
0812 19:04:36.028845 (+ 949us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:36.028884 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:36.028941 (+ 57us) server_negotiation.cc:407] Connection header received
0812 19:04:36.029922 (+ 981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:36.029951 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:36.030558 (+ 607us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:36.031002 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:36.031530 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:36.033117 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:36.034251 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:36.035022 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:36.035197 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:36.036152 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:36.036159 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:36.036180 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:04:36.036742 (+ 562us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:36.036825 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:04:36.036864 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:36.036969 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:36.037075 (+ 106us) server_negotiation.cc:299] Negotiation successful
0812 19:04:36.037397 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":156,"threads_started":1}
I20260812 19:04:36.042536 5123 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7096045f3b9341de9685ac2b82766fd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7096045f3b9341de9685ac2b82766fd4" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46115 } }
I20260812 19:04:36.042788 5123 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7096045f3b9341de9685ac2b82766fd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:36.042824 5123 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7096045f3b9341de9685ac2b82766fd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7096045f3b9341de9685ac2b82766fd4, State: Initialized, Role: FOLLOWER
I20260812 19:04:36.043282 5123 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7096045f3b9341de9685ac2b82766fd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7096045f3b9341de9685ac2b82766fd4" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46115 } }
I20260812 19:04:36.043426 5123 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7096045f3b9341de9685ac2b82766fd4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:04:36.043475 5123 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7096045f3b9341de9685ac2b82766fd4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:04:36.043555 5123 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7096045f3b9341de9685ac2b82766fd4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:36.044589 5123 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7096045f3b9341de9685ac2b82766fd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7096045f3b9341de9685ac2b82766fd4" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46115 } }
I20260812 19:04:36.044929 5123 leader_election.cc:302] T 00000000000000000000000000000000 P 7096045f3b9341de9685ac2b82766fd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7096045f3b9341de9685ac2b82766fd4; no voters:
I20260812 19:04:36.045228 5123 leader_election.cc:288] T 00000000000000000000000000000000 P 7096045f3b9341de9685ac2b82766fd4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:04:36.045408 5129 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7096045f3b9341de9685ac2b82766fd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:04:36.045706 5129 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7096045f3b9341de9685ac2b82766fd4 [term 1 LEADER]: Becoming Leader. State: Replica: 7096045f3b9341de9685ac2b82766fd4, State: Running, Role: LEADER
I20260812 19:04:36.046136 5129 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7096045f3b9341de9685ac2b82766fd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7096045f3b9341de9685ac2b82766fd4" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46115 } }
I20260812 19:04:36.046751 5123 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7096045f3b9341de9685ac2b82766fd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:04:36.047978 5131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7096045f3b9341de9685ac2b82766fd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7096045f3b9341de9685ac2b82766fd4. Latest consensus state: current_term: 1 leader_uuid: "7096045f3b9341de9685ac2b82766fd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7096045f3b9341de9685ac2b82766fd4" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46115 } } }
I20260812 19:04:36.048033 5130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7096045f3b9341de9685ac2b82766fd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7096045f3b9341de9685ac2b82766fd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7096045f3b9341de9685ac2b82766fd4" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46115 } } }
I20260812 19:04:36.048110 5131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7096045f3b9341de9685ac2b82766fd4 [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:36.048132 5130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7096045f3b9341de9685ac2b82766fd4 [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:36.048525 5139 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:04:36.052255 5139 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:04:36.058820 5139 catalog_manager.cc:1347] Generated new cluster ID: dca49e2b4bfa49c2b20aa466c3a71739
I20260812 19:04:36.058890 5139 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:04:36.074380 5139 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:04:36.075343 5139 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:04:36.082146 5139 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7096045f3b9341de9685ac2b82766fd4: Generated new TSK 0
I20260812 19:04:36.082911 5139 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:04:36.085886 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:0
--local_ip_for_outbound_sockets=127.25.130.1
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:46115
--builtin_ntp_servers=127.25.130.20:43329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 19:04:36.224165 5149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:36.224522 5149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:36.224565 5149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:36.228935 5149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:36.229022 5149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:36.229080 5149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:04:36.234208 5149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:43329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:46115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:36.235630 5149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:36.237365 5149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:36.248905 5158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:36.248914 5156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:36.248914 5155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:36.248987 5149 server_base.cc:1034] running on GCE node
I20260812 19:04:36.249732 5149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:36.250455 5149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:36.251639 5149 hybrid_clock.cc:648] HybridClock initialized: now 1786561476251615 us; error 38 us; skew 500 ppm
I20260812 19:04:36.253755 5149 webserver.cc:460] Webserver started at http://127.25.130.1:34151/ using document root <none> and password file <none>
I20260812 19:04:36.254431 5149 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:36.254514 5149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:36.254750 5149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:36.257027 5149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/data/instance:
uuid: "96ea93610a8c452491df4ae59652b360"
format_stamp: "Formatted at 2026-08-12 19:04:36 on dist-test-slave-49h3"
server_key: "45ae3ae2026bdc8051905d6a112107fe"
server_key_iv: "cabcda23b35850565f223d7cd0d4eeda"
server_key_version: "encryptionkey@0"
I20260812 19:04:36.257673 5149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance:
uuid: "96ea93610a8c452491df4ae59652b360"
format_stamp: "Formatted at 2026-08-12 19:04:36 on dist-test-slave-49h3"
server_key: "45ae3ae2026bdc8051905d6a112107fe"
server_key_iv: "cabcda23b35850565f223d7cd0d4eeda"
server_key_version: "encryptionkey@0"
I20260812 19:04:36.261925 5149 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 19:04:36.264858 5164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:36.266369 5149 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:04:36.266500 5149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "96ea93610a8c452491df4ae59652b360"
format_stamp: "Formatted at 2026-08-12 19:04:36 on dist-test-slave-49h3"
server_key: "45ae3ae2026bdc8051905d6a112107fe"
server_key_iv: "cabcda23b35850565f223d7cd0d4eeda"
server_key_version: "encryptionkey@0"
I20260812 19:04:36.266628 5149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:36.289600 5149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:36.290429 5149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:36.290638 5149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:36.291317 5149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:36.292438 5149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:36.292501 5149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:36.292559 5149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:36.292583 5149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:36.301798 5149 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:45049
I20260812 19:04:36.301841 5295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:45049 every 8 connection(s)
I20260812 19:04:36.302841 5149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:04:36.306262 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 5149
I20260812 19:04:36.306398 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance
I20260812 19:04:36.306768 26120 external_mini_cluster.cc:1428] Setting key 6f8410c82841f6aa7bba77403b0b2dd4
I20260812 19:04:36.309136 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:0
--local_ip_for_outbound_sockets=127.25.130.2
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:46115
--builtin_ntp_servers=127.25.130.20:43329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 19:04:36.313843 5126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:36.304885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:56287 (local address 127.25.130.62:46115)
0812 19:04:36.305069 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:36.305075 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:36.306467 (+ 1392us) server_negotiation.cc:407] Connection header received
0812 19:04:36.308182 (+ 1715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:36.308185 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:36.308235 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:36.308315 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:36.309605 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:36.310172 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:36.311688 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:36.312029 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:36.312144 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:36.313279 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:36.313283 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:36.313285 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:04:36.313302 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:36.313329 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:04:36.313339 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:36.313413 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:36.313585 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 19:04:36.313664 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 19:04:36.314811 5298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:36.305380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46115 (local address 127.25.130.1:56287)
0812 19:04:36.306272 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:36.306337 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:36.307952 (+ 1615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:36.308535 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:36.308546 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:36.308677 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:36.309428 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:36.309473 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:36.310308 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:36.310311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:36.311549 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:36.311561 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:36.312190 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:36.312193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:36.312348 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:36.312979 (+ 631us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:04:36.312981 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:36.313029 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:04:36.313105 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:36.313417 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:36.313422 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:36.313426 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 19:04:36.313576 (+ 150us) client_negotiation.cc:240] Negotiation successful
0812 19:04:36.313854 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":131,"threads_started":1}
I20260812 19:04:36.316133 5296 heartbeater.cc:347] Connected to a master server at 127.25.130.62:46115
I20260812 19:04:36.316390 5296 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:36.316975 5296 heartbeater.cc:511] Master 127.25.130.62:46115 requested a full tablet report, sending...
I20260812 19:04:36.318640 5079 ts_manager.cc:194] Registered new tserver with Master: 96ea93610a8c452491df4ae59652b360 (127.25.130.1:45049)
I20260812 19:04:36.319887 5079 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.130.1:56287
W20260812 19:04:36.432951 5299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:36.433257 5299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:36.433295 5299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:36.437427 5299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:36.437516 5299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:36.437572 5299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:04:36.442418 5299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:43329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:46115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:36.443581 5299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:36.445034 5299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:36.455320 5305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:36.455338 5308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:36.455385 5299 server_base.cc:1034] running on GCE node
W20260812 19:04:36.455353 5304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:36.456017 5299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:36.456743 5299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:36.457923 5299 hybrid_clock.cc:648] HybridClock initialized: now 1786561476457901 us; error 39 us; skew 500 ppm
I20260812 19:04:36.459988 5299 webserver.cc:460] Webserver started at http://127.25.130.2:39601/ using document root <none> and password file <none>
I20260812 19:04:36.460606 5299 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:36.460673 5299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:36.460855 5299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:36.463042 5299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/data/instance:
uuid: "65939e7417f041cd812cd394efcff0ec"
format_stamp: "Formatted at 2026-08-12 19:04:36 on dist-test-slave-49h3"
server_key: "1b5d6a69cc679d0f3113a387d3944e0e"
server_key_iv: "d33906085a49b2c072bb4d88ffb71d50"
server_key_version: "encryptionkey@0"
I20260812 19:04:36.463596 5299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance:
uuid: "65939e7417f041cd812cd394efcff0ec"
format_stamp: "Formatted at 2026-08-12 19:04:36 on dist-test-slave-49h3"
server_key: "1b5d6a69cc679d0f3113a387d3944e0e"
server_key_iv: "d33906085a49b2c072bb4d88ffb71d50"
server_key_version: "encryptionkey@0"
I20260812 19:04:36.467571 5299 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:04:36.470252 5314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:36.471587 5299 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:04:36.471699 5299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "65939e7417f041cd812cd394efcff0ec"
format_stamp: "Formatted at 2026-08-12 19:04:36 on dist-test-slave-49h3"
server_key: "1b5d6a69cc679d0f3113a387d3944e0e"
server_key_iv: "d33906085a49b2c072bb4d88ffb71d50"
server_key_version: "encryptionkey@0"
I20260812 19:04:36.471815 5299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:36.492506 5299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:36.493255 5299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:36.493434 5299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:36.494117 5299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:36.495128 5299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:36.495178 5299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:36.495234 5299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:36.495260 5299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:36.503991 5299 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:39763
I20260812 19:04:36.504024 5455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:39763 every 8 connection(s)
I20260812 19:04:36.504954 5299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:04:36.507776 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 5299
I20260812 19:04:36.507893 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance
I20260812 19:04:36.508180 26120 external_mini_cluster.cc:1428] Setting key 31774043e64db7251b3989adf9be6424
I20260812 19:04:36.510177 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:0
--local_ip_for_outbound_sockets=127.25.130.3
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:46115
--builtin_ntp_servers=127.25.130.20:43329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 19:04:36.514711 5126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:36.506891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:49851 (local address 127.25.130.62:46115)
0812 19:04:36.507076 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:36.507080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:36.508245 (+ 1165us) server_negotiation.cc:407] Connection header received
0812 19:04:36.509792 (+ 1547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:36.509796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:36.509852 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:36.509925 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:36.510919 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:36.511525 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:36.512701 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:36.512927 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:36.513015 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:36.514217 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:36.514220 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:36.514223 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:04:36.514240 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:36.514261 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:04:36.514269 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:36.514327 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:36.514483 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 19:04:36.514553 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 19:04:36.515661 5459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:36.507302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46115 (local address 127.25.130.2:49851)
0812 19:04:36.508062 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:36.508120 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:36.509558 (+ 1438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:36.510107 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:36.510115 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:36.510208 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:36.510773 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:36.510788 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:36.511658 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:36.511661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:36.512574 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:36.512585 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:36.513085 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:36.513089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:36.513268 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:36.513925 (+ 657us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:04:36.513927 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:36.513974 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:04:36.514051 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:36.514339 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:36.514350 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:36.514356 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 19:04:36.514472 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 19:04:36.514812 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":126,"threads_started":1}
I20260812 19:04:36.517052 5457 heartbeater.cc:347] Connected to a master server at 127.25.130.62:46115
I20260812 19:04:36.517308 5457 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:36.517903 5457 heartbeater.cc:511] Master 127.25.130.62:46115 requested a full tablet report, sending...
I20260812 19:04:36.519147 5079 ts_manager.cc:194] Registered new tserver with Master: 65939e7417f041cd812cd394efcff0ec (127.25.130.2:39763)
I20260812 19:04:36.519961 5079 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.130.2:49851
W20260812 19:04:36.630908 5460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:36.631201 5460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:36.631242 5460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:36.635404 5460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:36.635484 5460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:36.635542 5460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:04:36.640570 5460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:43329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:46115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:36.641790 5460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:36.643201 5460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:04:36.653386 5460 server_base.cc:1034] running on GCE node
W20260812 19:04:36.653363 5468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:36.653345 5465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:36.653345 5466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:36.654057 5460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:36.654698 5460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:36.655864 5460 hybrid_clock.cc:648] HybridClock initialized: now 1786561476655843 us; error 33 us; skew 500 ppm
I20260812 19:04:36.657804 5460 webserver.cc:460] Webserver started at http://127.25.130.3:37271/ using document root <none> and password file <none>
I20260812 19:04:36.658385 5460 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:36.658458 5460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:36.658696 5460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:36.660846 5460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/data/instance:
uuid: "75d99a31bb0b466dba986b5c623c4524"
format_stamp: "Formatted at 2026-08-12 19:04:36 on dist-test-slave-49h3"
server_key: "5cb8f5984f9c84c99dd46285261f8500"
server_key_iv: "d01125f1aca277cac42a11cd42f2e3aa"
server_key_version: "encryptionkey@0"
I20260812 19:04:36.661415 5460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance:
uuid: "75d99a31bb0b466dba986b5c623c4524"
format_stamp: "Formatted at 2026-08-12 19:04:36 on dist-test-slave-49h3"
server_key: "5cb8f5984f9c84c99dd46285261f8500"
server_key_iv: "d01125f1aca277cac42a11cd42f2e3aa"
server_key_version: "encryptionkey@0"
I20260812 19:04:36.665504 5460 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:04:36.668221 5475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:36.669575 5460 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:04:36.669701 5460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "75d99a31bb0b466dba986b5c623c4524"
format_stamp: "Formatted at 2026-08-12 19:04:36 on dist-test-slave-49h3"
server_key: "5cb8f5984f9c84c99dd46285261f8500"
server_key_iv: "d01125f1aca277cac42a11cd42f2e3aa"
server_key_version: "encryptionkey@0"
I20260812 19:04:36.669811 5460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:36.694779 5460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:36.695552 5460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:36.695741 5460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:36.696372 5460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:36.697388 5460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:36.697469 5460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:36.697540 5460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:36.697570 5460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:36.706312 5460 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:40691
I20260812 19:04:36.706337 5607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:40691 every 8 connection(s)
I20260812 19:04:36.707266 5460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:04:36.708887 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 5460
I20260812 19:04:36.708993 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance
I20260812 19:04:36.709271 26120 external_mini_cluster.cc:1428] Setting key 7692dfb265b6aee3b7fe48af0c35af2a
I20260812 19:04:36.716626 5126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:36.709148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:55143 (local address 127.25.130.62:46115)
0812 19:04:36.709331 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:36.709336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:36.710638 (+ 1302us) server_negotiation.cc:407] Connection header received
0812 19:04:36.712156 (+ 1518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:36.712159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:36.712207 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:36.712271 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:36.713148 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:36.713749 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:36.714872 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:36.715090 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:36.715162 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:36.716234 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:36.716237 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:36.716239 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:04:36.716251 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:36.716267 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:04:36.716274 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:36.716323 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:36.716443 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 19:04:36.716496 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 19:04:36.717644 5610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:36.709645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46115 (local address 127.25.130.3:55143)
0812 19:04:36.710494 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:36.710552 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:36.711959 (+ 1407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:36.712430 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:36.712437 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:36.712524 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:36.713037 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:36.713050 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:36.713855 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:36.713858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:36.714771 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:36.714782 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:36.715220 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:36.715222 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:36.715367 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:36.715982 (+ 615us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:04:36.715983 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:36.716026 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:04:36.716100 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:36.716332 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:36.716336 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:36.716340 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 19:04:36.716432 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 19:04:36.716682 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":133,"threads_started":1}
I20260812 19:04:36.718814 5608 heartbeater.cc:347] Connected to a master server at 127.25.130.62:46115
I20260812 19:04:36.719059 5608 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:36.719623 5608 heartbeater.cc:511] Master 127.25.130.62:46115 requested a full tablet report, sending...
I20260812 19:04:36.720690 5079 ts_manager.cc:194] Registered new tserver with Master: 75d99a31bb0b466dba986b5c623c4524 (127.25.130.3:40691)
I20260812 19:04:36.721261 5079 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.130.3:55143
I20260812 19:04:36.723152 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 19:04:36.726361 5126 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:36.725185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49588 (local address 127.25.130.62:46115)
0812 19:04:36.725334 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:36.725339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:36.725616 (+ 277us) server_negotiation.cc:407] Connection header received
0812 19:04:36.725775 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:36.725778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:36.725826 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:36.725902 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:36.726222 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49588: BlockingRecv error: recv got EOF from 127.0.0.1:49588 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 19:04:36.726521 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 5149
I20260812 19:04:36.742507 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 5299
I20260812 19:04:36.754191 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 5460
I20260812 19:04:36.764775 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 5043
2026-08-12T19:04:36Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (1002 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T19:04:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:04:36Z Disabled control of system clock
I20260812 19:04:36.798278 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:40121
--webserver_interface=127.25.130.62
--webserver_port=0
--builtin_ntp_servers=127.25.130.20:44347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:40121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 19:04:36.912819 5621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:36.913131 5621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:36.913170 5621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:36.917380 5621 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:04:36.917493 5621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:36.917508 5621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:36.917526 5621 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:04:36.917554 5621 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:04:36.922668 5621 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:44347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:40121
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:40121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:36.923974 5621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:36.925546 5621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:04:36.934455 5621 server_base.cc:1034] running on GCE node
W20260812 19:04:36.934397 5627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:36.934397 5629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:36.934409 5626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:36.935298 5621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:36.936482 5621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:36.937647 5621 hybrid_clock.cc:648] HybridClock initialized: now 1786561476937630 us; error 36 us; skew 500 ppm
I20260812 19:04:36.939564 5621 webserver.cc:460] Webserver started at http://127.25.130.62:38623/ using document root <none> and password file <none>
I20260812 19:04:36.940150 5621 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:36.940223 5621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:36.940444 5621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:36.942678 5621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "79ec82c4c3544beb950af5cbb2d8fbe0"
format_stamp: "Formatted at 2026-08-12 19:04:36 on dist-test-slave-49h3"
server_key: "49bd0a0dce3e68d082f25fb13924262a"
server_key_iv: "87f5b3d24d167b8f71ad29dda65db3fe"
server_key_version: "encryptionkey@0"
I20260812 19:04:36.943276 5621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "79ec82c4c3544beb950af5cbb2d8fbe0"
format_stamp: "Formatted at 2026-08-12 19:04:36 on dist-test-slave-49h3"
server_key: "49bd0a0dce3e68d082f25fb13924262a"
server_key_iv: "87f5b3d24d167b8f71ad29dda65db3fe"
server_key_version: "encryptionkey@0"
I20260812 19:04:36.947350 5621 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:04:36.949903 5638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:36.951267 5621 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:04:36.951400 5621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "79ec82c4c3544beb950af5cbb2d8fbe0"
format_stamp: "Formatted at 2026-08-12 19:04:36 on dist-test-slave-49h3"
server_key: "49bd0a0dce3e68d082f25fb13924262a"
server_key_iv: "87f5b3d24d167b8f71ad29dda65db3fe"
server_key_version: "encryptionkey@0"
I20260812 19:04:36.951516 5621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:36.956425 5621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:36.957122 5621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:36.957293 5621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:36.964629 5621 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:40121
I20260812 19:04:36.964659 5695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:40121 every 8 connection(s)
I20260812 19:04:36.965655 5621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:04:36.969275 5696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:36.974761 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 5621
I20260812 19:04:36.974905 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:04:36.975172 26120 external_mini_cluster.cc:1428] Setting key 63972027e41442faa8d8759b130e0c00
I20260812 19:04:36.975600 5696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ec82c4c3544beb950af5cbb2d8fbe0: Bootstrap starting.
I20260812 19:04:36.978330 5696 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79ec82c4c3544beb950af5cbb2d8fbe0: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:36.979274 5696 log.cc:824] T 00000000000000000000000000000000 P 79ec82c4c3544beb950af5cbb2d8fbe0: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:36.981571 5699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:36.976744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58722 (local address 127.25.130.62:40121)
0812 19:04:36.977405 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:36.977421 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:36.977494 (+ 73us) server_negotiation.cc:407] Connection header received
0812 19:04:36.978215 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:36.978234 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:36.978649 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:36.978952 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:36.979846 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:36.979853 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:36.979866 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:04:36.979877 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:36.979938 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:04:36.979955 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:36.980042 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:36.980281 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 19:04:36.980565 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":124,"threads_started":1}
I20260812 19:04:36.981923 5696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ec82c4c3544beb950af5cbb2d8fbe0: No bootstrap required, opened a new log
I20260812 19:04:36.985860 5696 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79ec82c4c3544beb950af5cbb2d8fbe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ec82c4c3544beb950af5cbb2d8fbe0" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40121 } }
I20260812 19:04:36.986120 5696 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79ec82c4c3544beb950af5cbb2d8fbe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:36.986171 5696 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79ec82c4c3544beb950af5cbb2d8fbe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79ec82c4c3544beb950af5cbb2d8fbe0, State: Initialized, Role: FOLLOWER
I20260812 19:04:36.986575 5696 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79ec82c4c3544beb950af5cbb2d8fbe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ec82c4c3544beb950af5cbb2d8fbe0" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40121 } }
I20260812 19:04:36.986716 5696 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79ec82c4c3544beb950af5cbb2d8fbe0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:04:36.986757 5696 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79ec82c4c3544beb950af5cbb2d8fbe0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:04:36.986831 5696 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79ec82c4c3544beb950af5cbb2d8fbe0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:36.987748 5696 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79ec82c4c3544beb950af5cbb2d8fbe0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ec82c4c3544beb950af5cbb2d8fbe0" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40121 } }
I20260812 19:04:36.988061 5696 leader_election.cc:302] T 00000000000000000000000000000000 P 79ec82c4c3544beb950af5cbb2d8fbe0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79ec82c4c3544beb950af5cbb2d8fbe0; no voters:
I20260812 19:04:36.988379 5696 leader_election.cc:288] T 00000000000000000000000000000000 P 79ec82c4c3544beb950af5cbb2d8fbe0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:04:36.988479 5703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79ec82c4c3544beb950af5cbb2d8fbe0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:04:36.988720 5703 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79ec82c4c3544beb950af5cbb2d8fbe0 [term 1 LEADER]: Becoming Leader. State: Replica: 79ec82c4c3544beb950af5cbb2d8fbe0, State: Running, Role: LEADER
I20260812 19:04:36.989096 5703 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79ec82c4c3544beb950af5cbb2d8fbe0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ec82c4c3544beb950af5cbb2d8fbe0" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40121 } }
I20260812 19:04:36.989763 5696 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79ec82c4c3544beb950af5cbb2d8fbe0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:04:36.990856 5704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ec82c4c3544beb950af5cbb2d8fbe0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79ec82c4c3544beb950af5cbb2d8fbe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ec82c4c3544beb950af5cbb2d8fbe0" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40121 } } }
I20260812 19:04:36.990887 5705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ec82c4c3544beb950af5cbb2d8fbe0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79ec82c4c3544beb950af5cbb2d8fbe0. Latest consensus state: current_term: 1 leader_uuid: "79ec82c4c3544beb950af5cbb2d8fbe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ec82c4c3544beb950af5cbb2d8fbe0" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40121 } } }
I20260812 19:04:36.990988 5704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ec82c4c3544beb950af5cbb2d8fbe0 [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:36.990994 5705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ec82c4c3544beb950af5cbb2d8fbe0 [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:36.991361 5713 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:04:36.994828 5713 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:04:37.001194 5713 catalog_manager.cc:1347] Generated new cluster ID: 900e72062840480f91f68910221fea2d
I20260812 19:04:37.001288 5713 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:04:37.014321 5713 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:04:37.014681 5713 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:04:37.028750 5713 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79ec82c4c3544beb950af5cbb2d8fbe0: Generated new TSK 0
I20260812 19:04:37.029918 5713 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:04:37.041790 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:0
--local_ip_for_outbound_sockets=127.25.130.1
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:40121
--builtin_ntp_servers=127.25.130.20:44347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 19:04:37.158844 5725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:37.159135 5725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:37.159178 5725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:37.163334 5725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:37.163410 5725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:37.163471 5725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:04:37.168529 5725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:44347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:40121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:37.169714 5725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:37.171154 5725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:37.180996 5731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:37.180994 5730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:37.181186 5725 server_base.cc:1034] running on GCE node
W20260812 19:04:37.181002 5735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:37.181751 5725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:37.182396 5725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:37.183558 5725 hybrid_clock.cc:648] HybridClock initialized: now 1786561477183540 us; error 33 us; skew 500 ppm
I20260812 19:04:37.185422 5725 webserver.cc:460] Webserver started at http://127.25.130.1:42007/ using document root <none> and password file <none>
I20260812 19:04:37.186080 5725 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:37.186154 5725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:37.186367 5725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:37.188464 5725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/data/instance:
uuid: "7f59f9196e0a457292b7a4bf7eeceeb4"
format_stamp: "Formatted at 2026-08-12 19:04:37 on dist-test-slave-49h3"
server_key: "a5b71a8323ee76e9b56e7f81d66d4f6f"
server_key_iv: "ab83424161a1ca550e7350a3eae9a504"
server_key_version: "encryptionkey@0"
I20260812 19:04:37.189054 5725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance:
uuid: "7f59f9196e0a457292b7a4bf7eeceeb4"
format_stamp: "Formatted at 2026-08-12 19:04:37 on dist-test-slave-49h3"
server_key: "a5b71a8323ee76e9b56e7f81d66d4f6f"
server_key_iv: "ab83424161a1ca550e7350a3eae9a504"
server_key_version: "encryptionkey@0"
I20260812 19:04:37.193085 5725 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 19:04:37.195839 5742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:37.197103 5725 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:04:37.197219 5725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "7f59f9196e0a457292b7a4bf7eeceeb4"
format_stamp: "Formatted at 2026-08-12 19:04:37 on dist-test-slave-49h3"
server_key: "a5b71a8323ee76e9b56e7f81d66d4f6f"
server_key_iv: "ab83424161a1ca550e7350a3eae9a504"
server_key_version: "encryptionkey@0"
I20260812 19:04:37.197335 5725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:37.202056 5725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:37.202708 5725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:37.202875 5725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:37.203451 5725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:37.204396 5725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:37.204444 5725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:37.204486 5725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:37.204514 5725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:37.213124 5725 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:39877
I20260812 19:04:37.213157 5876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:39877 every 8 connection(s)
I20260812 19:04:37.214087 5725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:04:37.219067 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 5725
I20260812 19:04:37.219182 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance
I20260812 19:04:37.219455 26120 external_mini_cluster.cc:1428] Setting key 8f9d30a909c45cc39f4455abfc476545
I20260812 19:04:37.220261 5699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:37.215886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:42171 (local address 127.25.130.62:40121)
0812 19:04:37.216061 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:37.216066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:37.217140 (+ 1074us) server_negotiation.cc:407] Connection header received
0812 19:04:37.218486 (+ 1346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:37.218488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:37.218528 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:37.218603 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:37.219773 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:37.219777 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:37.219779 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:04:37.219795 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:37.219820 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:04:37.219829 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:37.219910 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:37.220098 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 19:04:37.220152 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 19:04:37.221336 5881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:37.216319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:40121 (local address 127.25.130.1:42171)
0812 19:04:37.216986 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:37.217033 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:37.218297 (+ 1264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:37.218761 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:37.218768 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:37.218857 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:37.219465 (+ 608us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:04:37.219467 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:37.219508 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:04:37.219617 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:37.219926 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:37.219938 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:37.219944 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 19:04:37.220122 (+ 178us) client_negotiation.cc:240] Negotiation successful
0812 19:04:37.220382 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":108,"threads_started":1}
I20260812 19:04:37.221501 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:0
--local_ip_for_outbound_sockets=127.25.130.2
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:40121
--builtin_ntp_servers=127.25.130.20:44347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 19:04:37.222633 5877 heartbeater.cc:347] Connected to a master server at 127.25.130.62:40121
I20260812 19:04:37.222856 5877 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:37.223263 5877 heartbeater.cc:511] Master 127.25.130.62:40121 requested a full tablet report, sending...
I20260812 19:04:37.224922 5657 ts_manager.cc:194] Registered new tserver with Master: 7f59f9196e0a457292b7a4bf7eeceeb4 (127.25.130.1:39877)
W20260812 19:04:37.335456 5882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:37.335769 5882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:37.335829 5882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:37.340070 5882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:37.340149 5882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:37.340230 5882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:04:37.345285 5882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:44347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:40121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:37.346539 5882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:37.347965 5882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:37.357875 5888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:37.357925 5889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:37.357969 5882 server_base.cc:1034] running on GCE node
W20260812 19:04:37.357892 5891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:37.358583 5882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:37.359308 5882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:37.360491 5882 hybrid_clock.cc:648] HybridClock initialized: now 1786561477360450 us; error 57 us; skew 500 ppm
I20260812 19:04:37.362515 5882 webserver.cc:460] Webserver started at http://127.25.130.2:37701/ using document root <none> and password file <none>
I20260812 19:04:37.363114 5882 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:37.363185 5882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:37.363385 5882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:37.365724 5882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/data/instance:
uuid: "8753bc9f09f64e1b92b2630833f4c0d9"
format_stamp: "Formatted at 2026-08-12 19:04:37 on dist-test-slave-49h3"
server_key: "309e5a6cc85c9e058399c01516bc395c"
server_key_iv: "b358785fc20585fd305cdb56e54b29b7"
server_key_version: "encryptionkey@0"
I20260812 19:04:37.366376 5882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance:
uuid: "8753bc9f09f64e1b92b2630833f4c0d9"
format_stamp: "Formatted at 2026-08-12 19:04:37 on dist-test-slave-49h3"
server_key: "309e5a6cc85c9e058399c01516bc395c"
server_key_iv: "b358785fc20585fd305cdb56e54b29b7"
server_key_version: "encryptionkey@0"
I20260812 19:04:37.370200 5882 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 19:04:37.372810 5900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:37.374131 5882 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260812 19:04:37.374250 5882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "8753bc9f09f64e1b92b2630833f4c0d9"
format_stamp: "Formatted at 2026-08-12 19:04:37 on dist-test-slave-49h3"
server_key: "309e5a6cc85c9e058399c01516bc395c"
server_key_iv: "b358785fc20585fd305cdb56e54b29b7"
server_key_version: "encryptionkey@0"
I20260812 19:04:37.374361 5882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:37.379148 5882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:37.379825 5882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:37.379992 5882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:37.380621 5882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:37.381655 5882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:37.381707 5882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:37.381762 5882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:37.381788 5882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:37.390690 5882 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:37691
I20260812 19:04:37.390717 6036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:37691 every 8 connection(s)
I20260812 19:04:37.391675 5882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:04:37.398016 5699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:37.393564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:46543 (local address 127.25.130.62:40121)
0812 19:04:37.393694 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:37.393698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:37.394867 (+ 1169us) server_negotiation.cc:407] Connection header received
0812 19:04:37.396240 (+ 1373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:37.396243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:37.396294 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:37.396370 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:37.397533 (+ 1163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:37.397537 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:37.397540 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:04:37.397556 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:37.397577 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:04:37.397586 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:37.397671 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:37.397808 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 19:04:37.397859 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 19:04:37.398959 6041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:37.394006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:40121 (local address 127.25.130.2:46543)
0812 19:04:37.394712 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:37.394757 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:37.396009 (+ 1252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:37.396566 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:37.396574 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:37.396678 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:37.397243 (+ 565us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:04:37.397244 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:37.397287 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:04:37.397360 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:37.397664 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:37.397674 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:37.397681 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 19:04:37.397852 (+ 171us) client_negotiation.cc:240] Negotiation successful
0812 19:04:37.398120 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":127,"threads_started":1}
I20260812 19:04:37.399075 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 5882
I20260812 19:04:37.399197 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance
I20260812 19:04:37.399492 26120 external_mini_cluster.cc:1428] Setting key 1ab47046e276b42fa9b3ea3f3c961376
I20260812 19:04:37.400837 6037 heartbeater.cc:347] Connected to a master server at 127.25.130.62:40121
I20260812 19:04:37.401129 6037 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:37.401710 6037 heartbeater.cc:511] Master 127.25.130.62:40121 requested a full tablet report, sending...
I20260812 19:04:37.401770 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:0
--local_ip_for_outbound_sockets=127.25.130.3
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:40121
--builtin_ntp_servers=127.25.130.20:44347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 19:04:37.402870 5657 ts_manager.cc:194] Registered new tserver with Master: 8753bc9f09f64e1b92b2630833f4c0d9 (127.25.130.2:37691)
W20260812 19:04:37.516287 6042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:37.516566 6042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:37.516606 6042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:37.520658 6042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:37.520730 6042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:37.520789 6042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:04:37.525660 6042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:44347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:40121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:37.526801 6042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:37.528285 6042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:04:37.538314 6042 server_base.cc:1034] running on GCE node
W20260812 19:04:37.538233 6050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:37.538249 6053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:37.538245 6051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:37.538923 6042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:37.539565 6042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:37.540726 6042 hybrid_clock.cc:648] HybridClock initialized: now 1786561477540698 us; error 42 us; skew 500 ppm
I20260812 19:04:37.542668 6042 webserver.cc:460] Webserver started at http://127.25.130.3:42337/ using document root <none> and password file <none>
I20260812 19:04:37.543270 6042 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:37.543339 6042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:37.543536 6042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:37.545809 6042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/data/instance:
uuid: "82eccce656824af5bb0981160983d9ae"
format_stamp: "Formatted at 2026-08-12 19:04:37 on dist-test-slave-49h3"
server_key: "624d370e43d4e9c02874ce02cafea184"
server_key_iv: "4d839ccde9bd87be57c71af7bcd9081d"
server_key_version: "encryptionkey@0"
I20260812 19:04:37.546406 6042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance:
uuid: "82eccce656824af5bb0981160983d9ae"
format_stamp: "Formatted at 2026-08-12 19:04:37 on dist-test-slave-49h3"
server_key: "624d370e43d4e9c02874ce02cafea184"
server_key_iv: "4d839ccde9bd87be57c71af7bcd9081d"
server_key_version: "encryptionkey@0"
I20260812 19:04:37.550415 6042 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:04:37.553121 6061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:37.554477 6042 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 19:04:37.554637 6042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "82eccce656824af5bb0981160983d9ae"
format_stamp: "Formatted at 2026-08-12 19:04:37 on dist-test-slave-49h3"
server_key: "624d370e43d4e9c02874ce02cafea184"
server_key_iv: "4d839ccde9bd87be57c71af7bcd9081d"
server_key_version: "encryptionkey@0"
I20260812 19:04:37.554769 6042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:37.559508 6042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:37.560158 6042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:37.560326 6042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:37.560899 6042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:37.561923 6042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:37.561975 6042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:37.562029 6042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:37.562055 6042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:37.570845 6042 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:33289
I20260812 19:04:37.570869 6193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:33289 every 8 connection(s)
I20260812 19:04:37.571806 6042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:04:37.578127 5699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:37.573703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:40229 (local address 127.25.130.62:40121)
0812 19:04:37.573842 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:37.573846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:37.574976 (+ 1130us) server_negotiation.cc:407] Connection header received
0812 19:04:37.576336 (+ 1360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:37.576340 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:37.576392 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:37.576470 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:37.577644 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:37.577648 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:37.577650 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:04:37.577665 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:37.577687 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:04:37.577696 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:37.577781 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:37.577925 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 19:04:37.577977 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 19:04:37.579157 6196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:37.574124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:40121 (local address 127.25.130.3:40229)
0812 19:04:37.574823 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:37.574866 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:37.576111 (+ 1245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:37.576658 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:37.576665 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:37.576758 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:37.577298 (+ 540us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:04:37.577300 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:37.577363 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:04:37.577491 (+ 128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:37.577769 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:37.577778 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:37.577784 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 19:04:37.577963 (+ 179us) client_negotiation.cc:240] Negotiation successful
0812 19:04:37.578271 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":119,"threads_started":1}
I20260812 19:04:37.579205 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 6042
I20260812 19:04:37.579310 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance
I20260812 19:04:37.579584 26120 external_mini_cluster.cc:1428] Setting key 48671d2469fec3ea025ee428e0d48bae
I20260812 19:04:37.580379 6194 heartbeater.cc:347] Connected to a master server at 127.25.130.62:40121
I20260812 19:04:37.580642 6194 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:37.581146 6194 heartbeater.cc:511] Master 127.25.130.62:40121 requested a full tablet report, sending...
I20260812 19:04:37.582231 5657 ts_manager.cc:194] Registered new tserver with Master: 82eccce656824af5bb0981160983d9ae (127.25.130.3:33289)
I20260812 19:04:37.593691 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 19:04:37.596611 5699 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:04:37.595701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58738 (local address 127.25.130.62:40121)
0812 19:04:37.595846 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:37.595851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:37.595970 (+ 119us) server_negotiation.cc:407] Connection header received
0812 19:04:37.596085 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:37.596087 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:37.596123 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:37.596190 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:37.596460 (+ 270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58738: BlockingRecv error: recv got EOF from 127.0.0.1:58738 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 19:04:37.597092 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 5725
I20260812 19:04:37.607088 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 5882
I20260812 19:04:37.614835 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 6042
I20260812 19:04:37.627676 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 5621
2026-08-12T19:04:37Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (863 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:04:37 dist-test-slave-49h3 krb5kdc[6210](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:04:37 dist-test-slave-49h3 krb5kdc[6210](info): set up 2 sockets
Aug 12 19:04:37 dist-test-slave-49h3 krb5kdc[6210](info): commencing operation
krb5kdc: starting...
W20260812 19:04:38.703215 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:04:38 dist-test-slave-49h3 krb5kdc[6210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:04:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:04:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:04:38.840929 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:39397
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:34225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:39397
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:04:38.975433 6226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:38.975765 6226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:38.975834 6226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:38.980057 6226 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:04:38.980135 6226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:38.980163 6226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:38.980185 6226 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:04:38.980206 6226 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:04:38.985867 6226 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:34225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:39397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:39397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:38.987196 6226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:38.988448 6226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:38.998327 6235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:38.998342 6233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:38.998308 6237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:38.998746 6226 server_base.cc:1034] running on GCE node
I20260812 19:04:38.999383 6226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:39.000761 6226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:39.001945 6226 hybrid_clock.cc:648] HybridClock initialized: now 1786561479001925 us; error 38 us; skew 500 ppm
Aug 12 19:04:39 dist-test-slave-49h3 krb5kdc[6210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561479, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:39.010128 6226 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:04:39.011322 6226 webserver.cc:460] Webserver started at http://127.25.130.62:44583/ using document root <none> and password file <none>
I20260812 19:04:39.012023 6226 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:39.012100 6226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:39.012363 6226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:39.014989 6226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "4a0d3003006540a78d869a732a04009a"
format_stamp: "Formatted at 2026-08-12 19:04:39 on dist-test-slave-49h3"
server_key: "a6a6e14f2b5a603899dcf01d55a0d7d6"
server_key_iv: "dc179aefb008b4a0af236131b6105071"
server_key_version: "encryptionkey@0"
I20260812 19:04:39.015681 6226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "4a0d3003006540a78d869a732a04009a"
format_stamp: "Formatted at 2026-08-12 19:04:39 on dist-test-slave-49h3"
server_key: "a6a6e14f2b5a603899dcf01d55a0d7d6"
server_key_iv: "dc179aefb008b4a0af236131b6105071"
server_key_version: "encryptionkey@0"
I20260812 19:04:39.020406 6226 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 19:04:39.023407 6245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:39.024933 6226 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 19:04:39.025091 6226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "4a0d3003006540a78d869a732a04009a"
format_stamp: "Formatted at 2026-08-12 19:04:39 on dist-test-slave-49h3"
server_key: "a6a6e14f2b5a603899dcf01d55a0d7d6"
server_key_iv: "dc179aefb008b4a0af236131b6105071"
server_key_version: "encryptionkey@0"
I20260812 19:04:39.025209 6226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:39.051095 6226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:39.054677 6226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:39.054893 6226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:39.063504 6226 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:39397
I20260812 19:04:39.063539 6308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:39397 every 8 connection(s)
I20260812 19:04:39.064635 6226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:04:39.068089 6309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:39.071053 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 6226
I20260812 19:04:39.071175 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:04:39.071518 26120 external_mini_cluster.cc:1428] Setting key 8c8ccb6501704a12b3f6da377f8afdfc
I20260812 19:04:39.075181 6309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a0d3003006540a78d869a732a04009a: Bootstrap starting.
I20260812 19:04:39.078436 6309 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a0d3003006540a78d869a732a04009a: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:39.079465 6309 log.cc:824] T 00000000000000000000000000000000 P 4a0d3003006540a78d869a732a04009a: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:39.081869 6309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a0d3003006540a78d869a732a04009a: No bootstrap required, opened a new log
Aug 12 19:04:39 dist-test-slave-49h3 krb5kdc[6210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:04:39.086771 6309 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a0d3003006540a78d869a732a04009a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0d3003006540a78d869a732a04009a" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39397 } }
I20260812 19:04:39.087096 6309 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a0d3003006540a78d869a732a04009a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:39.087165 6309 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a0d3003006540a78d869a732a04009a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a0d3003006540a78d869a732a04009a, State: Initialized, Role: FOLLOWER
I20260812 19:04:39.087705 6309 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a0d3003006540a78d869a732a04009a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0d3003006540a78d869a732a04009a" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39397 } }
I20260812 19:04:39.087872 6309 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a0d3003006540a78d869a732a04009a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:04:39.087940 6309 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a0d3003006540a78d869a732a04009a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:04:39.088037 6309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a0d3003006540a78d869a732a04009a [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:39.088208 6312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:39.073048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60266 (local address 127.25.130.62:39397)
0812 19:04:39.073575 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:39.073600 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:39.073760 (+ 160us) server_negotiation.cc:407] Connection header received
0812 19:04:39.074694 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:39.074719 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:39.075175 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:39.075589 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:39.076309 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:39.077924 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:39.078837 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:39.079584 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:39.079757 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:39.082559 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:39.082566 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:39.082584 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:39.082616 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:39.084614 (+ 1998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:39.085061 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:39.085068 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:39.085074 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:39.085233 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:39.085610 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:39.085613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:39.085615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:39.086213 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:39.086442 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:39.086723 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 19:04:39.087059 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":129,"threads_started":1}
I20260812 19:04:39.089149 6309 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a0d3003006540a78d869a732a04009a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0d3003006540a78d869a732a04009a" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39397 } }
I20260812 19:04:39.089570 6309 leader_election.cc:302] T 00000000000000000000000000000000 P 4a0d3003006540a78d869a732a04009a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a0d3003006540a78d869a732a04009a; no voters:
I20260812 19:04:39.089915 6309 leader_election.cc:288] T 00000000000000000000000000000000 P 4a0d3003006540a78d869a732a04009a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:04:39.090032 6314 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a0d3003006540a78d869a732a04009a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:04:39.090215 6314 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a0d3003006540a78d869a732a04009a [term 1 LEADER]: Becoming Leader. State: Replica: 4a0d3003006540a78d869a732a04009a, State: Running, Role: LEADER
I20260812 19:04:39.090634 6314 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a0d3003006540a78d869a732a04009a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0d3003006540a78d869a732a04009a" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39397 } }
I20260812 19:04:39.091427 6309 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a0d3003006540a78d869a732a04009a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:04:39.092614 6315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a0d3003006540a78d869a732a04009a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a0d3003006540a78d869a732a04009a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0d3003006540a78d869a732a04009a" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39397 } } }
I20260812 19:04:39.092753 6315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a0d3003006540a78d869a732a04009a [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:39.093065 6316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a0d3003006540a78d869a732a04009a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a0d3003006540a78d869a732a04009a. Latest consensus state: current_term: 1 leader_uuid: "4a0d3003006540a78d869a732a04009a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a0d3003006540a78d869a732a04009a" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39397 } } }
I20260812 19:04:39.093155 6316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a0d3003006540a78d869a732a04009a [sys.catalog]: This master's current role is: LEADER
I20260812 19:04:39.093199 6325 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:04:39.097059 6325 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:04:39.102995 6325 catalog_manager.cc:1347] Generated new cluster ID: ad34e29ec1a5433891b3c7c97cd2cef6
I20260812 19:04:39.103062 6325 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:04:39.116604 6325 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:04:39.118080 6325 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:04:39.132922 6325 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a0d3003006540a78d869a732a04009a: Generated new TSK 0
I20260812 19:04:39.134025 6325 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
I20260812 19:04:39.174989 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:0
--local_ip_for_outbound_sockets=127.25.130.1
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:39397
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:34225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:04:39.309176 6341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:39.309513 6341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:39.309580 6341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:39.313642 6341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:39.313711 6341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:39.313804 6341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:04:39.319355 6341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:34225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:39397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:39.320626 6341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:39.321871 6341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:39.332520 6350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:39.332540 6348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:39.332542 6347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:39.333058 6341 server_base.cc:1034] running on GCE node
I20260812 19:04:39.333549 6341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:39.334306 6341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:39.335466 6341 hybrid_clock.cc:648] HybridClock initialized: now 1786561479335449 us; error 33 us; skew 500 ppm
Aug 12 19:04:39 dist-test-slave-49h3 krb5kdc[6210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561479, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:39.338475 6341 init.cc:377] Logged in from keytab as kudu/127.25.130.1@KRBTEST.COM (short username kudu)
I20260812 19:04:39.339462 6341 webserver.cc:460] Webserver started at http://127.25.130.1:33271/ using document root <none> and password file <none>
I20260812 19:04:39.340085 6341 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:39.340155 6341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:39.340386 6341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:39.342694 6341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/data/instance:
uuid: "f40d8be653a042499b1129c364e40093"
format_stamp: "Formatted at 2026-08-12 19:04:39 on dist-test-slave-49h3"
server_key: "76f4e6dd4690cc2529b8b66aa46d33a4"
server_key_iv: "6a029aeb14318f13e8550f873ec40b1c"
server_key_version: "encryptionkey@0"
I20260812 19:04:39.343289 6341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance:
uuid: "f40d8be653a042499b1129c364e40093"
format_stamp: "Formatted at 2026-08-12 19:04:39 on dist-test-slave-49h3"
server_key: "76f4e6dd4690cc2529b8b66aa46d33a4"
server_key_iv: "6a029aeb14318f13e8550f873ec40b1c"
server_key_version: "encryptionkey@0"
I20260812 19:04:39.347414 6341 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:04:39.350126 6358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:39.351493 6341 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:04:39.351625 6341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "f40d8be653a042499b1129c364e40093"
format_stamp: "Formatted at 2026-08-12 19:04:39 on dist-test-slave-49h3"
server_key: "76f4e6dd4690cc2529b8b66aa46d33a4"
server_key_iv: "6a029aeb14318f13e8550f873ec40b1c"
server_key_version: "encryptionkey@0"
I20260812 19:04:39.351761 6341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:39.366997 6341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:39.369944 6341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:39.370133 6341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:39.370754 6341 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:39.371618 6341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:39.371680 6341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:39.371752 6341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:39.371778 6341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:39.380911 6341 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:35521
I20260812 19:04:39.380936 6493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:35521 every 8 connection(s)
I20260812 19:04:39.381922 6341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:04:39.384356 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 6341
I20260812 19:04:39.384471 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance
I20260812 19:04:39.384802 26120 external_mini_cluster.cc:1428] Setting key 5cdeccf76cbae60f03929c408e47198e
Aug 12 19:04:39 dist-test-slave-49h3 krb5kdc[6210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561479, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.130.2@KRBTEST.COM; defaulting to no policy
I20260812 19:04:39.396858 6312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:39.383854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:53261 (local address 127.25.130.62:39397)
0812 19:04:39.383993 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:39.383997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:39.384937 (+ 940us) server_negotiation.cc:407] Connection header received
0812 19:04:39.386577 (+ 1640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:39.386580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:39.386637 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:39.386709 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:39.387990 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:39.388552 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:39.389777 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:39.390012 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:39.390101 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:39.393159 (+ 3058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:39.393163 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:39.393165 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:39.393195 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:39.394617 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:39.395162 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:39.395166 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:39.395167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:39.395251 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:39.395588 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:39.395592 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:39.395593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:39.395830 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:39.395921 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:39.396483 (+ 562us) server_negotiation.cc:299] Negotiation successful
0812 19:04:39.396664 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 19:04:39.397810 6497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:39.384226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:39397 (local address 127.25.130.1:53261)
0812 19:04:39.384771 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:39.384829 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:39.386351 (+ 1522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:39.386916 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:39.386924 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:39.387316 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:39.387838 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:39.387853 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:39.388684 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:39.388687 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:39.389654 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:39.389665 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:39.390165 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:39.390170 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:39.390344 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:39.390923 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:39.390941 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:39.392911 (+ 1970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:39.394746 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:39.394754 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:39.394775 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:39.395060 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:39.395338 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:39.395341 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:39.395343 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:39.395498 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:39.395931 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:39.395939 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:39.396236 (+ 297us) client_negotiation.cc:769] Sending connection context
0812 19:04:39.396423 (+ 187us) client_negotiation.cc:240] Negotiation successful
0812 19:04:39.396799 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":139,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.2@KRBTEST.COM" created.
I20260812 19:04:39.399243 6494 heartbeater.cc:347] Connected to a master server at 127.25.130.62:39397
I20260812 19:04:39.399497 6494 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:39.400040 6494 heartbeater.cc:511] Master 127.25.130.62:39397 requested a full tablet report, sending...
I20260812 19:04:39.401829 6266 ts_manager.cc:194] Registered new tserver with Master: f40d8be653a042499b1129c364e40093 (127.25.130.1:35521)
I20260812 19:04:39.403085 6266 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.1@KRBTEST.COM'} at 127.25.130.1:53261
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
I20260812 19:04:39.414664 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:0
--local_ip_for_outbound_sockets=127.25.130.2
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:39397
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:34225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:04:39.534766 6503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:39.535063 6503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:39.535125 6503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:39.539300 6503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:39.539381 6503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:39.539463 6503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:04:39.545092 6503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:34225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:39397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:39.546326 6503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:39.547472 6503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:39.557029 6509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:39.557063 6508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:39.557029 6511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:39.557317 6503 server_base.cc:1034] running on GCE node
I20260812 19:04:39.557783 6503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:39.558408 6503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:39.559578 6503 hybrid_clock.cc:648] HybridClock initialized: now 1786561479559551 us; error 38 us; skew 500 ppm
Aug 12 19:04:39 dist-test-slave-49h3 krb5kdc[6210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561479, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:39.562417 6503 init.cc:377] Logged in from keytab as kudu/127.25.130.2@KRBTEST.COM (short username kudu)
I20260812 19:04:39.563290 6503 webserver.cc:460] Webserver started at http://127.25.130.2:42611/ using document root <none> and password file <none>
I20260812 19:04:39.563814 6503 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:39.563872 6503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:39.564046 6503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:39.566130 6503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/data/instance:
uuid: "0b6dbe9839664fd18024f721fb6b59e4"
format_stamp: "Formatted at 2026-08-12 19:04:39 on dist-test-slave-49h3"
server_key: "6578ba130d1a696b884e69d2d0e0d155"
server_key_iv: "66763718fc332e3b4bbb2fc996b39404"
server_key_version: "encryptionkey@0"
I20260812 19:04:39.566660 6503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance:
uuid: "0b6dbe9839664fd18024f721fb6b59e4"
format_stamp: "Formatted at 2026-08-12 19:04:39 on dist-test-slave-49h3"
server_key: "6578ba130d1a696b884e69d2d0e0d155"
server_key_iv: "66763718fc332e3b4bbb2fc996b39404"
server_key_version: "encryptionkey@0"
I20260812 19:04:39.570346 6503 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 19:04:39.572667 6521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:39.573880 6503 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:04:39.573983 6503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "0b6dbe9839664fd18024f721fb6b59e4"
format_stamp: "Formatted at 2026-08-12 19:04:39 on dist-test-slave-49h3"
server_key: "6578ba130d1a696b884e69d2d0e0d155"
server_key_iv: "66763718fc332e3b4bbb2fc996b39404"
server_key_version: "encryptionkey@0"
I20260812 19:04:39.574072 6503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:39.588302 6503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:39.591490 6503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:39.591691 6503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:39.592304 6503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:39.593227 6503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:39.593284 6503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:39.593338 6503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:39.593366 6503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:39.601857 6503 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:34925
I20260812 19:04:39.601879 6655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:34925 every 8 connection(s)
I20260812 19:04:39.602770 6503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:04:39.612095 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 6503
I20260812 19:04:39.612226 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance
I20260812 19:04:39.612500 26120 external_mini_cluster.cc:1428] Setting key 4f52903927304341a26443f8facafb7f
Aug 12 19:04:39 dist-test-slave-49h3 krb5kdc[6210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561479, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:04:39.616876 6312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:39.604745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:45725 (local address 127.25.130.62:39397)
0812 19:04:39.604874 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:39.604878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:39.605774 (+ 896us) server_negotiation.cc:407] Connection header received
0812 19:04:39.607189 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:39.607193 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:39.607254 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:39.607329 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:39.608574 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:39.609109 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:39.610308 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:39.610549 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:39.610644 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:39.613333 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:39.613336 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:39.613339 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:39.613364 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:39.614798 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:39.615319 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:39.615323 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:39.615324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:39.615411 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:39.615737 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:39.615740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:39.615742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:39.615941 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:39.616030 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:39.616564 (+ 534us) server_negotiation.cc:299] Negotiation successful
0812 19:04:39.616708 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 19:04:39.617813 6659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:39.605111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:39397 (local address 127.25.130.2:45725)
0812 19:04:39.605621 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:39.605667 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:39.606960 (+ 1293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:39.607522 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:39.607529 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:39.607908 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:39.608416 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:39.608431 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:39.609235 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:39.609238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:39.610181 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:39.610192 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:39.610701 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:39.610706 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:39.610870 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:39.611474 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:39.611492 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:39.613179 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:39.614915 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:39.614922 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:39.614940 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:39.615221 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:39.615494 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:39.615497 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:39.615499 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:39.615647 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:39.616044 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:39.616051 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:39.616322 (+ 271us) client_negotiation.cc:769] Sending connection context
0812 19:04:39.616498 (+ 176us) client_negotiation.cc:240] Negotiation successful
0812 19:04:39.616843 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":124,"threads_started":1}
I20260812 19:04:39.618957 6656 heartbeater.cc:347] Connected to a master server at 127.25.130.62:39397
I20260812 19:04:39.619241 6656 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:39.619887 6656 heartbeater.cc:511] Master 127.25.130.62:39397 requested a full tablet report, sending...
I20260812 19:04:39.621125 6266 ts_manager.cc:194] Registered new tserver with Master: 0b6dbe9839664fd18024f721fb6b59e4 (127.25.130.2:34925)
I20260812 19:04:39.621956 6266 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.2@KRBTEST.COM'} at 127.25.130.2:45725
WARNING: no policy specified for kudu/127.25.130.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
I20260812 19:04:39.640066 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:0
--local_ip_for_outbound_sockets=127.25.130.3
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:39397
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:34225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:04:39.751976 6663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:04:39.752260 6663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:04:39.752307 6663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:04:39.756486 6663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:04:39.756560 6663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:04:39.756636 6663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:04:39.762121 6663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:34225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:39397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:04:39.763265 6663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:04:39.764346 6663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:04:39.773815 6669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:04:39.773819 6670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:39.773916 6663 server_base.cc:1034] running on GCE node
W20260812 19:04:39.773818 6672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:04:39.774483 6663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:04:39.775122 6663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:04:39.776314 6663 hybrid_clock.cc:648] HybridClock initialized: now 1786561479776282 us; error 40 us; skew 500 ppm
Aug 12 19:04:39 dist-test-slave-49h3 krb5kdc[6210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561479, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:04:39.779165 6663 init.cc:377] Logged in from keytab as kudu/127.25.130.3@KRBTEST.COM (short username kudu)
I20260812 19:04:39.780057 6663 webserver.cc:460] Webserver started at http://127.25.130.3:38449/ using document root <none> and password file <none>
I20260812 19:04:39.780613 6663 fs_manager.cc:359] Metadata directory not provided
I20260812 19:04:39.780682 6663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:04:39.780900 6663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:04:39.783052 6663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/data/instance:
uuid: "ed2d0fe22b1e4d6db651d786de4b6f45"
format_stamp: "Formatted at 2026-08-12 19:04:39 on dist-test-slave-49h3"
server_key: "dc7bd2c5fe731daf2453f329b75fc2dc"
server_key_iv: "07c21b443435455b04c626b06b33b67e"
server_key_version: "encryptionkey@0"
I20260812 19:04:39.783654 6663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance:
uuid: "ed2d0fe22b1e4d6db651d786de4b6f45"
format_stamp: "Formatted at 2026-08-12 19:04:39 on dist-test-slave-49h3"
server_key: "dc7bd2c5fe731daf2453f329b75fc2dc"
server_key_iv: "07c21b443435455b04c626b06b33b67e"
server_key_version: "encryptionkey@0"
I20260812 19:04:39.787688 6663 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:04:39.790307 6679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:39.791599 6663 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:04:39.791720 6663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "ed2d0fe22b1e4d6db651d786de4b6f45"
format_stamp: "Formatted at 2026-08-12 19:04:39 on dist-test-slave-49h3"
server_key: "dc7bd2c5fe731daf2453f329b75fc2dc"
server_key_iv: "07c21b443435455b04c626b06b33b67e"
server_key_version: "encryptionkey@0"
I20260812 19:04:39.791832 6663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:04:39.812296 6663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:04:39.815330 6663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:04:39.815522 6663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:04:39.816147 6663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:04:39.817013 6663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:04:39.817066 6663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:39.817121 6663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:04:39.817150 6663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:04:39.825881 6663 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:38873
I20260812 19:04:39.825907 6814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:38873 every 8 connection(s)
I20260812 19:04:39.826798 6663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:04:39.827314 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 6663
I20260812 19:04:39.827435 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance
I20260812 19:04:39.827751 26120 external_mini_cluster.cc:1428] Setting key f651f8efd45937850e79d9039d75e8f6
Aug 12 19:04:39 dist-test-slave-49h3 krb5kdc[6210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561479, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:04:39.840027 6312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:39.828798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:33063 (local address 127.25.130.62:39397)
0812 19:04:39.828947 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:39.828952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:39.829947 (+ 995us) server_negotiation.cc:407] Connection header received
0812 19:04:39.831407 (+ 1460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:39.831410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:39.831456 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:39.831519 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:39.832630 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:39.833175 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:39.834312 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:39.834533 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:39.834618 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:39.836915 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:39.836918 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:39.836920 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:39.836947 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:39.838289 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:39.838738 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:39.838740 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:39.838742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:39.838817 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:39.839092 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:39.839094 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:39.839095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:39.839285 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:39.839358 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:39.839794 (+ 436us) server_negotiation.cc:299] Negotiation successful
0812 19:04:39.839898 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 19:04:39.840930 6819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:39.829257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:39397 (local address 127.25.130.3:33063)
0812 19:04:39.829796 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:39.829854 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:39.831219 (+ 1365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:39.831659 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:39.831666 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:39.832013 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:04:39.832500 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:39.832514 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:39.833282 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:39.833285 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:39.834197 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:39.834207 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:39.834652 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:39.834655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:39.834796 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:39.835307 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:04:39.835323 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:04:39.836753 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:04:39.838386 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:39.838391 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:39.838403 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:39.838661 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:39.838879 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:04:39.838881 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:04:39.838882 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:04:39.839007 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:04:39.839358 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:04:39.839362 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:04:39.839607 (+ 245us) client_negotiation.cc:769] Sending connection context
0812 19:04:39.839764 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 19:04:39.840050 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":143,"threads_started":1}
I20260812 19:04:39.842031 6816 heartbeater.cc:347] Connected to a master server at 127.25.130.62:39397
I20260812 19:04:39.842253 6816 heartbeater.cc:464] Registering TS with master...
I20260812 19:04:39.842727 6816 heartbeater.cc:511] Master 127.25.130.62:39397 requested a full tablet report, sending...
I20260812 19:04:39.843703 6266 ts_manager.cc:194] Registered new tserver with Master: ed2d0fe22b1e4d6db651d786de4b6f45 (127.25.130.3:38873)
I20260812 19:04:39.844254 6266 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.3@KRBTEST.COM'} at 127.25.130.3:33063
I20260812 19:04:39.853703 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:04:39.863436 6312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:39.855645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60274 (local address 127.25.130.62:39397)
0812 19:04:39.855779 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:39.855785 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:39.856008 (+ 223us) server_negotiation.cc:407] Connection header received
0812 19:04:39.856143 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:39.856146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:39.856191 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:39.856260 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:04:39.856939 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:39.857518 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:39.858330 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:39.858546 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:39.858622 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:39.860201 (+ 1579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:04:39.860205 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:04:39.860207 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:04:39.860238 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:04:39.861911 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:39.862326 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:39.862328 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:39.862329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:39.862409 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:04:39.862601 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:04:39.862603 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:04:39.862604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:04:39.862834 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:04:39.862934 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:39.863201 (+ 267us) server_negotiation.cc:299] Negotiation successful
0812 19:04:39.863307 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 19:04:39.866477 6266 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60274:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:04:39.869745 6266 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:04:39.884603 6830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:39.878874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56522 (local address 127.25.130.2:34925)
0812 19:04:39.879632 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:39.879646 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:39.879671 (+ 25us) server_negotiation.cc:407] Connection header received
0812 19:04:39.879753 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:39.879758 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:39.879986 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:39.880142 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:39.880469 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:39.881417 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:39.883011 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:39.883707 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:39.883807 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:39.883969 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:39.884363 (+ 394us) server_negotiation.cc:299] Negotiation successful
0812 19:04:39.884460 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":178,"threads_started":1}
I20260812 19:04:39.884611 6828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:39.878717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:34925 (local address 127.0.0.1:56522)
0812 19:04:39.879094 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:39.879146 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:39.879276 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:39.880193 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:39.880200 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:39.880221 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:39.880360 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:39.880368 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:39.881559 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:39.881563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:39.882877 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:39.882887 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:39.884037 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:39.884041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:39.884156 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:39.884195 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 19:04:39.884312 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 19:04:39.884449 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":1,"thread_start_us":96,"threads_started":1}
I20260812 19:04:39.885581 6829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:39.878940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:38873 (local address 127.0.0.1:51134)
0812 19:04:39.879497 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:39.879511 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:39.879600 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:39.880830 (+ 1230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:39.880833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:39.880854 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:39.880957 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:39.880963 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:39.882269 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:39.882272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:39.883672 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:39.883683 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:39.885155 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:39.885159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:39.885251 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:39.885276 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 19:04:39.885353 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 19:04:39.885424 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"mutex_wait_us":96,"thread_start_us":93,"threads_started":1}
I20260812 19:04:39.885704 6834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:39.879907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51134 (local address 127.25.130.3:38873)
0812 19:04:39.880372 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:39.880388 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:39.880410 (+ 22us) server_negotiation.cc:407] Connection header received
0812 19:04:39.880485 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:39.880490 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:39.880682 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:39.880838 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:39.881052 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:39.882100 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:39.883801 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:39.884595 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:39.884713 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:39.884860 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:39.885418 (+ 558us) server_negotiation.cc:299] Negotiation successful
0812 19:04:39.885545 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":147,"threads_started":1}
I20260812 19:04:39.886366 6832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:39.879462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:35521 (local address 127.0.0.1:39518)
0812 19:04:39.879963 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:39.879975 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:39.880058 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:39.881037 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:39.881040 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:39.881058 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:39.881145 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:39.881151 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:39.882239 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:39.882242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:39.883673 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:39.883683 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:39.885965 (+ 2282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:39.885968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:39.886078 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:39.886104 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 19:04:39.886173 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 19:04:39.886246 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":145,"thread_start_us":97,"threads_started":1}
I20260812 19:04:39.886898 6833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:39.879641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39518 (local address 127.25.130.1:35521)
0812 19:04:39.880491 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:39.880506 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:39.880525 (+ 19us) server_negotiation.cc:407] Connection header received
0812 19:04:39.880593 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:39.880597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:39.880754 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:39.880877 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:39.881273 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:39.882107 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:39.884930 (+ 2823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:39.885604 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:39.885705 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:39.886593 (+ 888us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:39.886649 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 19:04:39.886752 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":753,"thread_start_us":110,"threads_started":1}
I20260812 19:04:39.888180 6574 tablet_service.cc:1467] Processing CreateTablet for tablet 1837100523fa412f801604812280c2bd (DEFAULT_TABLE table=test-table [id=d4139be326e2464db1b8f931cf914e30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:04:39.888638 6736 tablet_service.cc:1467] Processing CreateTablet for tablet 1837100523fa412f801604812280c2bd (DEFAULT_TABLE table=test-table [id=d4139be326e2464db1b8f931cf914e30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:04:39.888911 6412 tablet_service.cc:1467] Processing CreateTablet for tablet 1837100523fa412f801604812280c2bd (DEFAULT_TABLE table=test-table [id=d4139be326e2464db1b8f931cf914e30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:04:39.889493 6574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1837100523fa412f801604812280c2bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:39.889628 6736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1837100523fa412f801604812280c2bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:39.889809 6412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1837100523fa412f801604812280c2bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:04:39.896829 6835 tablet_bootstrap.cc:492] T 1837100523fa412f801604812280c2bd P ed2d0fe22b1e4d6db651d786de4b6f45: Bootstrap starting.
I20260812 19:04:39.898005 6837 tablet_bootstrap.cc:492] T 1837100523fa412f801604812280c2bd P f40d8be653a042499b1129c364e40093: Bootstrap starting.
I20260812 19:04:39.898238 6836 tablet_bootstrap.cc:492] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4: Bootstrap starting.
I20260812 19:04:39.899976 6835 tablet_bootstrap.cc:654] T 1837100523fa412f801604812280c2bd P ed2d0fe22b1e4d6db651d786de4b6f45: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:39.900135 6837 tablet_bootstrap.cc:654] T 1837100523fa412f801604812280c2bd P f40d8be653a042499b1129c364e40093: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:39.900295 6836 tablet_bootstrap.cc:654] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4: Neither blocks nor log segments found. Creating new log.
I20260812 19:04:39.900981 6837 log.cc:824] T 1837100523fa412f801604812280c2bd P f40d8be653a042499b1129c364e40093: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:39.900981 6835 log.cc:824] T 1837100523fa412f801604812280c2bd P ed2d0fe22b1e4d6db651d786de4b6f45: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:39.901099 6836 log.cc:824] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4: Log is configured to *not* fsync() on all Append() calls
I20260812 19:04:39.903141 6836 tablet_bootstrap.cc:492] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4: No bootstrap required, opened a new log
I20260812 19:04:39.903244 6835 tablet_bootstrap.cc:492] T 1837100523fa412f801604812280c2bd P ed2d0fe22b1e4d6db651d786de4b6f45: No bootstrap required, opened a new log
I20260812 19:04:39.903354 6836 ts_tablet_manager.cc:1397] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 19:04:39.903406 6837 tablet_bootstrap.cc:492] T 1837100523fa412f801604812280c2bd P f40d8be653a042499b1129c364e40093: No bootstrap required, opened a new log
I20260812 19:04:39.903441 6835 ts_tablet_manager.cc:1397] T 1837100523fa412f801604812280c2bd P ed2d0fe22b1e4d6db651d786de4b6f45: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 19:04:39.903563 6837 ts_tablet_manager.cc:1397] T 1837100523fa412f801604812280c2bd P f40d8be653a042499b1129c364e40093: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 19:04:39.906438 6836 raft_consensus.cc:348] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6dbe9839664fd18024f721fb6b59e4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 34925 } } peers { permanent_uuid: "f40d8be653a042499b1129c364e40093" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 35521 } } peers { permanent_uuid: "ed2d0fe22b1e4d6db651d786de4b6f45" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38873 } }
I20260812 19:04:39.906689 6836 raft_consensus.cc:374] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:39.906744 6836 raft_consensus.cc:729] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b6dbe9839664fd18024f721fb6b59e4, State: Initialized, Role: FOLLOWER
I20260812 19:04:39.907178 6836 consensus_queue.cc:260] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6dbe9839664fd18024f721fb6b59e4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 34925 } } peers { permanent_uuid: "f40d8be653a042499b1129c364e40093" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 35521 } } peers { permanent_uuid: "ed2d0fe22b1e4d6db651d786de4b6f45" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38873 } }
I20260812 19:04:39.907236 6835 raft_consensus.cc:348] T 1837100523fa412f801604812280c2bd P ed2d0fe22b1e4d6db651d786de4b6f45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6dbe9839664fd18024f721fb6b59e4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 34925 } } peers { permanent_uuid: "f40d8be653a042499b1129c364e40093" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 35521 } } peers { permanent_uuid: "ed2d0fe22b1e4d6db651d786de4b6f45" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38873 } }
I20260812 19:04:39.907490 6835 raft_consensus.cc:374] T 1837100523fa412f801604812280c2bd P ed2d0fe22b1e4d6db651d786de4b6f45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:39.907554 6835 raft_consensus.cc:729] T 1837100523fa412f801604812280c2bd P ed2d0fe22b1e4d6db651d786de4b6f45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed2d0fe22b1e4d6db651d786de4b6f45, State: Initialized, Role: FOLLOWER
I20260812 19:04:39.907740 6837 raft_consensus.cc:348] T 1837100523fa412f801604812280c2bd P f40d8be653a042499b1129c364e40093 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6dbe9839664fd18024f721fb6b59e4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 34925 } } peers { permanent_uuid: "f40d8be653a042499b1129c364e40093" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 35521 } } peers { permanent_uuid: "ed2d0fe22b1e4d6db651d786de4b6f45" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38873 } }
I20260812 19:04:39.907958 6656 heartbeater.cc:503] Master 127.25.130.62:39397 was elected leader, sending a full tablet report...
I20260812 19:04:39.907976 6837 raft_consensus.cc:374] T 1837100523fa412f801604812280c2bd P f40d8be653a042499b1129c364e40093 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:04:39.908023 6837 raft_consensus.cc:729] T 1837100523fa412f801604812280c2bd P f40d8be653a042499b1129c364e40093 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f40d8be653a042499b1129c364e40093, State: Initialized, Role: FOLLOWER
I20260812 19:04:39.908010 6835 consensus_queue.cc:260] T 1837100523fa412f801604812280c2bd P ed2d0fe22b1e4d6db651d786de4b6f45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6dbe9839664fd18024f721fb6b59e4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 34925 } } peers { permanent_uuid: "f40d8be653a042499b1129c364e40093" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 35521 } } peers { permanent_uuid: "ed2d0fe22b1e4d6db651d786de4b6f45" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38873 } }
I20260812 19:04:39.908459 6837 consensus_queue.cc:260] T 1837100523fa412f801604812280c2bd P f40d8be653a042499b1129c364e40093 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6dbe9839664fd18024f721fb6b59e4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 34925 } } peers { permanent_uuid: "f40d8be653a042499b1129c364e40093" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 35521 } } peers { permanent_uuid: "ed2d0fe22b1e4d6db651d786de4b6f45" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38873 } }
I20260812 19:04:39.909054 6835 ts_tablet_manager.cc:1428] T 1837100523fa412f801604812280c2bd P ed2d0fe22b1e4d6db651d786de4b6f45: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 19:04:39.909260 6836 ts_tablet_manager.cc:1428] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260812 19:04:39.909339 6837 ts_tablet_manager.cc:1428] T 1837100523fa412f801604812280c2bd P f40d8be653a042499b1129c364e40093: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 19:04:39.909475 6816 heartbeater.cc:503] Master 127.25.130.62:39397 was elected leader, sending a full tablet report...
I20260812 19:04:39.909680 6494 heartbeater.cc:503] Master 127.25.130.62:39397 was elected leader, sending a full tablet report...
W20260812 19:04:40.077874 6818 tablet.cc:2369] T 1837100523fa412f801604812280c2bd P ed2d0fe22b1e4d6db651d786de4b6f45: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:04:40.084059 6843 raft_consensus.cc:482] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:04:40.084287 6843 raft_consensus.cc:504] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6dbe9839664fd18024f721fb6b59e4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 34925 } } peers { permanent_uuid: "f40d8be653a042499b1129c364e40093" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 35521 } } peers { permanent_uuid: "ed2d0fe22b1e4d6db651d786de4b6f45" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38873 } }
I20260812 19:04:40.085556 6843 leader_election.cc:288] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f40d8be653a042499b1129c364e40093 (127.25.130.1:35521), ed2d0fe22b1e4d6db651d786de4b6f45 (127.25.130.3:38873)
I20260812 19:04:40.089857 6833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:40.085965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:52005 (local address 127.25.130.1:35521)
0812 19:04:40.086149 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:40.086153 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:40.086169 (+ 16us) server_negotiation.cc:407] Connection header received
0812 19:04:40.086225 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:40.086227 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:40.086270 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:40.086333 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:40.086610 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:40.087274 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:40.088722 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:40.089253 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:40.089375 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:40.089434 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:40.089670 (+ 236us) server_negotiation.cc:299] Negotiation successful
0812 19:04:40.089744 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260812 19:04:40.089861 6659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:40.085788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:35521 (local address 127.25.130.2:52005)
0812 19:04:40.085982 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:40.085997 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:40.086120 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:40.086355 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:40.086358 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:40.086375 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:40.086493 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:40.086500 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:40.087391 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:40.087393 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:40.088582 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:40.088594 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:40.089394 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:40.089397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:40.089532 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:40.089555 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 19:04:40.089641 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 19:04:40.089720 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":1}
I20260812 19:04:40.090000 6846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:40.085817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:38873 (local address 127.25.130.2:49231)
0812 19:04:40.086219 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:40.086230 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 19:04:40.086354 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:04:40.086741 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:04:40.086743 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:04:40.086758 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:04:40.086853 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:40.086857 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:40.087695 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:40.087699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:40.088826 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:04:40.088833 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:04:40.089638 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:04:40.089641 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:04:40.089743 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:40.089764 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 19:04:40.089835 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 19:04:40.089893 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":28,"thread_start_us":92,"threads_started":1}
I20260812 19:04:40.090037 6834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:40.085970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:49231 (local address 127.25.130.3:38873)
0812 19:04:40.086121 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:40.086126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:40.086324 (+ 198us) server_negotiation.cc:407] Connection header received
0812 19:04:40.086505 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:40.086509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:40.086563 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:40.086614 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:04:40.086966 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:40.087582 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:40.088929 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:40.089534 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:40.089626 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:40.089672 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:40.089858 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 19:04:40.089931 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 19:04:40.090633 6437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1837100523fa412f801604812280c2bd" candidate_uuid: "0b6dbe9839664fd18024f721fb6b59e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f40d8be653a042499b1129c364e40093" is_pre_election: true
I20260812 19:04:40.090626 6760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1837100523fa412f801604812280c2bd" candidate_uuid: "0b6dbe9839664fd18024f721fb6b59e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed2d0fe22b1e4d6db651d786de4b6f45" is_pre_election: true
I20260812 19:04:40.090914 6760 raft_consensus.cc:2454] T 1837100523fa412f801604812280c2bd P ed2d0fe22b1e4d6db651d786de4b6f45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b6dbe9839664fd18024f721fb6b59e4 in term 0.
I20260812 19:04:40.090917 6437 raft_consensus.cc:2454] T 1837100523fa412f801604812280c2bd P f40d8be653a042499b1129c364e40093 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b6dbe9839664fd18024f721fb6b59e4 in term 0.
I20260812 19:04:40.091398 6525 leader_election.cc:302] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0b6dbe9839664fd18024f721fb6b59e4, ed2d0fe22b1e4d6db651d786de4b6f45, f40d8be653a042499b1129c364e40093; no voters:
I20260812 19:04:40.091671 6843 raft_consensus.cc:2781] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:04:40.091791 6843 raft_consensus.cc:482] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:04:40.091838 6843 raft_consensus.cc:3037] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:40.093007 6843 raft_consensus.cc:504] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6dbe9839664fd18024f721fb6b59e4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 34925 } } peers { permanent_uuid: "f40d8be653a042499b1129c364e40093" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 35521 } } peers { permanent_uuid: "ed2d0fe22b1e4d6db651d786de4b6f45" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38873 } }
I20260812 19:04:40.093366 6843 leader_election.cc:288] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4 [CANDIDATE]: Term 1 election: Requested vote from peers f40d8be653a042499b1129c364e40093 (127.25.130.1:35521), ed2d0fe22b1e4d6db651d786de4b6f45 (127.25.130.3:38873)
I20260812 19:04:40.093797 6437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1837100523fa412f801604812280c2bd" candidate_uuid: "0b6dbe9839664fd18024f721fb6b59e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f40d8be653a042499b1129c364e40093"
I20260812 19:04:40.093792 6760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1837100523fa412f801604812280c2bd" candidate_uuid: "0b6dbe9839664fd18024f721fb6b59e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed2d0fe22b1e4d6db651d786de4b6f45"
I20260812 19:04:40.093930 6760 raft_consensus.cc:3037] T 1837100523fa412f801604812280c2bd P ed2d0fe22b1e4d6db651d786de4b6f45 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:40.093930 6437 raft_consensus.cc:3037] T 1837100523fa412f801604812280c2bd P f40d8be653a042499b1129c364e40093 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:04:40.095271 6437 raft_consensus.cc:2454] T 1837100523fa412f801604812280c2bd P f40d8be653a042499b1129c364e40093 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b6dbe9839664fd18024f721fb6b59e4 in term 1.
I20260812 19:04:40.095271 6760 raft_consensus.cc:2454] T 1837100523fa412f801604812280c2bd P ed2d0fe22b1e4d6db651d786de4b6f45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b6dbe9839664fd18024f721fb6b59e4 in term 1.
I20260812 19:04:40.095577 6525 leader_election.cc:302] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b6dbe9839664fd18024f721fb6b59e4, ed2d0fe22b1e4d6db651d786de4b6f45; no voters:
I20260812 19:04:40.095746 6843 raft_consensus.cc:2781] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:04:40.096009 6843 raft_consensus.cc:686] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4 [term 1 LEADER]: Becoming Leader. State: Replica: 0b6dbe9839664fd18024f721fb6b59e4, State: Running, Role: LEADER
I20260812 19:04:40.096396 6843 consensus_queue.cc:237] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6dbe9839664fd18024f721fb6b59e4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 34925 } } peers { permanent_uuid: "f40d8be653a042499b1129c364e40093" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 35521 } } peers { permanent_uuid: "ed2d0fe22b1e4d6db651d786de4b6f45" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38873 } }
I20260812 19:04:40.099746 6264 catalog_manager.cc:5495] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b6dbe9839664fd18024f721fb6b59e4 (127.25.130.2). New cstate: current_term: 1 leader_uuid: "0b6dbe9839664fd18024f721fb6b59e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6dbe9839664fd18024f721fb6b59e4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 34925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f40d8be653a042499b1129c364e40093" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 35521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed2d0fe22b1e4d6db651d786de4b6f45" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38873 } health_report { overall_health: UNKNOWN } } }
W20260812 19:04:40.103978 6657 tablet.cc:2369] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:04:40.121367 6830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:40.117873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56536 (local address 127.25.130.2:34925)
0812 19:04:40.117998 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 19:04:40.118001 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:04:40.118013 (+ 12us) server_negotiation.cc:407] Connection header received
0812 19:04:40.118092 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:04:40.118094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:04:40.118153 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:04:40.118243 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:04:40.118559 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:40.119073 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:40.119931 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:04:40.120166 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:04:40.120243 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:04:40.120542 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:04:40.120991 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:04:40.121090 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 19:04:40.121166 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 19:04:40.121220 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 19:04:40.127826 6760 raft_consensus.cc:1261] T 1837100523fa412f801604812280c2bd P ed2d0fe22b1e4d6db651d786de4b6f45 [term 1 FOLLOWER]: Refusing update from remote peer 0b6dbe9839664fd18024f721fb6b59e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:04:40.127826 6437 raft_consensus.cc:1261] T 1837100523fa412f801604812280c2bd P f40d8be653a042499b1129c364e40093 [term 1 FOLLOWER]: Refusing update from remote peer 0b6dbe9839664fd18024f721fb6b59e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:04:40.128589 6847 consensus_queue.cc:1035] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed2d0fe22b1e4d6db651d786de4b6f45" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:04:40.128791 6843 consensus_queue.cc:1035] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f40d8be653a042499b1129c364e40093" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 35521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 19:04:40.134112 6495 tablet.cc:2369] T 1837100523fa412f801604812280c2bd P f40d8be653a042499b1129c364e40093: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:04:40.136896 6855 mvcc.cc:205] Tried to move back new op lower bound from 7317755822595239936 to 7317755822477627392. Current Snapshot: MvccSnapshot[applied={T|T < 7317755822595239936}]
I20260812 19:04:40.137404 6854 mvcc.cc:205] Tried to move back new op lower bound from 7317755822595239936 to 7317755822477627392. Current Snapshot: MvccSnapshot[applied={T|T < 7317755822595239936}]
I20260812 19:04:40.145823 6265 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60274:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:04:40.145991 6265 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60274:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:04:40.148540 6265 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4a0d3003006540a78d869a732a04009a: Sending DeleteTablet for 3 replicas of tablet 1837100523fa412f801604812280c2bd
I20260812 19:04:40.149372 6574 tablet_service.cc:1514] Processing DeleteTablet for tablet 1837100523fa412f801604812280c2bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:04:40 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:56522
I20260812 19:04:40.149572 6412 tablet_service.cc:1514] Processing DeleteTablet for tablet 1837100523fa412f801604812280c2bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:04:40 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:39518
I20260812 19:04:40.149974 6736 tablet_service.cc:1514] Processing DeleteTablet for tablet 1837100523fa412f801604812280c2bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:04:40 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:51134
I20260812 19:04:40.150029 6864 tablet_replica.cc:330] T 1837100523fa412f801604812280c2bd P f40d8be653a042499b1129c364e40093: stopping tablet replica
I20260812 19:04:40.150393 6864 raft_consensus.cc:2229] T 1837100523fa412f801604812280c2bd P f40d8be653a042499b1129c364e40093 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:04:40.150478 6862 tablet_replica.cc:330] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4: stopping tablet replica
I20260812 19:04:40.150626 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 6341
I20260812 19:04:40.151080 6862 raft_consensus.cc:2229] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4 [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:04:40.151469 6862 raft_consensus.cc:2258] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:04:40.154551 6862 ts_tablet_manager.cc:1905] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:04:40.155539 6865 tablet_replica.cc:330] T 1837100523fa412f801604812280c2bd P ed2d0fe22b1e4d6db651d786de4b6f45: stopping tablet replica
I20260812 19:04:40.155915 6865 raft_consensus.cc:2229] T 1837100523fa412f801604812280c2bd P ed2d0fe22b1e4d6db651d786de4b6f45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:04:40.156314 6865 raft_consensus.cc:2258] T 1837100523fa412f801604812280c2bd P ed2d0fe22b1e4d6db651d786de4b6f45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:04:40.157395 6862 ts_tablet_manager.cc:1918] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:04:40.157517 6862 log.cc:1196] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-1/wal/wals/1837100523fa412f801604812280c2bd
I20260812 19:04:40.157826 6862 ts_tablet_manager.cc:1939] T 1837100523fa412f801604812280c2bd P 0b6dbe9839664fd18024f721fb6b59e4: Deleting consensus metadata
I20260812 19:04:40.158226 6865 ts_tablet_manager.cc:1905] T 1837100523fa412f801604812280c2bd P ed2d0fe22b1e4d6db651d786de4b6f45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:04:40.159083 6251 catalog_manager.cc:4841] TS 0b6dbe9839664fd18024f721fb6b59e4 (127.25.130.2:34925): tablet 1837100523fa412f801604812280c2bd (table test-table [id=d4139be326e2464db1b8f931cf914e30]) successfully deleted
W20260812 19:04:40.160284 6251 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.130.1:35521 (error 108)
I20260812 19:04:40.160333 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 6503
I20260812 19:04:40.161053 6865 ts_tablet_manager.cc:1918] T 1837100523fa412f801604812280c2bd P ed2d0fe22b1e4d6db651d786de4b6f45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:04:40.161212 6865 log.cc:1196] T 1837100523fa412f801604812280c2bd P ed2d0fe22b1e4d6db651d786de4b6f45: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561415988404-26120-0/minicluster-data/ts-2/wal/wals/1837100523fa412f801604812280c2bd
I20260812 19:04:40.161554 6865 ts_tablet_manager.cc:1939] T 1837100523fa412f801604812280c2bd P ed2d0fe22b1e4d6db651d786de4b6f45: Deleting consensus metadata
I20260812 19:04:40.162781 6249 catalog_manager.cc:4841] TS ed2d0fe22b1e4d6db651d786de4b6f45 (127.25.130.3:38873): tablet 1837100523fa412f801604812280c2bd (table test-table [id=d4139be326e2464db1b8f931cf914e30]) successfully deleted
I20260812 19:04:40.163535 6829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:04:40.163051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:35521 (local address 127.0.0.1:39530)
0812 19:04:40.163356 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0812 19:04:40.163437 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.1:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218}
W20260812 19:04:40.163748 6251 catalog_manager.cc:4584] TS f40d8be653a042499b1129c364e40093 (127.25.130.1:35521): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1837100523fa412f801604812280c2bd: Network error: Client connection negotiation failed: client connection to 127.25.130.1:35521: connect: Connection refused (error 111)
I20260812 19:04:40.168915 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 6663
I20260812 19:04:40.176450 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 6226
2026-08-12T19:04:40Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2548 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:04:40 dist-test-slave-49h3 krb5kdc[6869](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:04:40 dist-test-slave-49h3 krb5kdc[6869](info): set up 2 sockets
Aug 12 19:04:40 dist-test-slave-49h3 krb5kdc[6869](info): commencing operation
krb5kdc: starting...
W20260812 19:04:41.245518 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:04:41 dist-test-slave-49h3 krb5kdc[6869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561481, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:04:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:04:41Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.25.130.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.25.130.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.25.130.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/rangeradmin_127.25.130.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.25.130.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/rangeradmin_127.25.130.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.25.130.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.25.130.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.25.130.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/rangerlookup_127.25.130.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.25.130.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/rangerlookup_127.25.130.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.25.130.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.25.130.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.25.130.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/HTTP_127.25.130.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.25.130.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/HTTP_127.25.130.20@KRBTEST.COM.keytab.
I20260812 19:04:41.423982 26120 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskt2JiBs/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/postgres -l logfile start
2026-08-12 19:04:43.821 UTC [6905] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 19:04:43.821 UTC [6905] LOG: listening on IPv4 address "127.25.130.20", port 38741
2026-08-12 19:04:43.824 UTC [6905] LOG: listening on Unix socket "/tmp/.s.PGSQL.38741"
2026-08-12 19:04:43.845 UTC [6908] LOG: database system was shut down at 2026-08-12 19:04:42 UTC
2026-08-12 19:04:43.848 UTC [6905] LOG: database system is ready to accept connections
I20260812 19:04:44.827193 26120 mini_postgres.cc:96] Postgres bound to 38741
2026-08-12 19:04:44.835 UTC [6919] FATAL: database "slave" does not exist
127.25.130.20:38741 - accepting connections
I20260812 19:04:44.836287 26120 mini_ranger.cc:161] Starting Ranger...
I20260812 19:04:44.853569 26120 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 19:04:45.278859 26120 mini_ranger.cc:87] Created ranger Postgres database
I20260812 19:04:45.278961 26120 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ranger-admin
2026-08-12 19:04:45,627 [I] DB FLAVOR :POSTGRES
2026-08-12 19:04:45,627 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 19:04:45,627 [I] Checking connection..
2026-08-12 19:04:45,628 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:38741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 19:04:45,893 [I] Checking connection passed.
2026-08-12 19:04:45,893 [I] --------- Verifying version history table ---------
2026-08-12 19:04:45,893 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:38741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 19:04:46,157 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 19:04:46,158 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:38741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 19:04:46,401 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 19:04:46,401 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 19:04:46,402 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:38741/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 19:04:46.656 UTC [6995] WARNING: there is no transaction in progress
2026-08-12 19:04:46,664 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 19:04:46,664 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:38741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 19:04:46,929 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 19:04:46,929 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 19:04:46,929 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:38741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 19:04:47,189 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:38741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 19:04:47,450 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:38741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-49h3.c.gcp-upstream.internal','N') ;"
2026-08-12 19:04:47,699 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:38741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 19:04:47,944 [I] Table x_portal_user does not exist in database ranger
2026-08-12 19:04:47,945 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:38741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 19:04:48,199 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 19:04:48,199 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:38741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 19:04:48,464 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:38741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 19:04:48,726 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 19:04:48,726 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:38741/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 19:04:49.709 UTC [7127] WARNING: there is no transaction in progress
2026-08-12 19:04:49.725 UTC [7127] WARNING: there is no transaction in progress
2026-08-12 19:04:49.740 UTC [7127] WARNING: there is no transaction in progress
2026-08-12 19:04:49.756 UTC [7127] WARNING: there is no transaction in progress
2026-08-12 19:04:49.771 UTC [7127] WARNING: there is no transaction in progress
2026-08-12 19:04:49.789 UTC [7127] WARNING: there is no transaction in progress
2026-08-12 19:04:49.854 UTC [7127] WARNING: there is no transaction in progress
2026-08-12 19:04:49.861 UTC [7127] WARNING: there is no transaction in progress
2026-08-12 19:04:49.875 UTC [7127] WARNING: there is no transaction in progress
2026-08-12 19:04:49.886 UTC [7127] WARNING: there is no transaction in progress
2026-08-12 19:04:49.898 UTC [7127] WARNING: there is no transaction in progress
2026-08-12 19:04:49.912 UTC [7127] WARNING: there is no transaction in progress
2026-08-12 19:04:49.923 UTC [7127] WARNING: there is no transaction in progress
2026-08-12 19:04:49.930 UTC [7127] WARNING: there is no transaction in progress
2026-08-12 19:04:49.938 UTC [7127] WARNING: there is no transaction in progress
2026-08-12 19:04:49.945 UTC [7127] WARNING: there is no transaction in progress
2026-08-12 19:04:50,599 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 19:04:50,599 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:38741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 19:04:50,850 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 19:04:50,850 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:38741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 19:04:51,093 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 19:04:51,094 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:38741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 19:04:51,359 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 19:04:51,359 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:38741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 19:04:51,615 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:38741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 19:04:51,887 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:38741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-49h3.c.gcp-upstream.internal';"
2026-08-12 19:04:52,151 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 19:04:52,151 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:38741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 19:04:52,414 [I] DB_PATCHES have already been applied
I20260812 19:04:52.421356 26120 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ranger-admin:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskt2JiBs/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 19:04:52.421471 26120 mini_ranger.cc:193] Using host: 127.25.130.20
I20260812 19:04:52.427558 26120 mini_ranger.cc:227] Ranger admin URL: http://127.25.130.20:38373
Aug 12, 2026 7:04:52 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 7:04:52 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 7:04:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 7:04:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 7:04:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 7:04:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 7:04:53 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 19:04:53.574959 26120 mini_ranger.cc:160] Time spent starting Ranger: real 8.739s user 0.005s sys 0.010s
Aug 12, 2026 7:05:13 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 7:05:13 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 7:05:13 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 7:05:13 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 7:05:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 7:05:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 7:05:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 7:05:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 7:05:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 7:05:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 7:05:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 7:05:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 7:05:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 7:05:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 7:05:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 7:05:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 7:05:13 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 7:05:14 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 19:05:15.132676 26120 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.25.130.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.25.130.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.25.130.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/rangerkms_127.25.130.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.25.130.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/rangerkms_127.25.130.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.25.130.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.25.130.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.25.130.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/HTTP_127.25.130.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.25.130.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/HTTP_127.25.130.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 19:05:15 dist-test-slave-49h3 krb5kdc[6869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561515, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 19:05:15.224799 26120 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 19:05:15.236303 26120 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 19:05:16.032433 26120 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 19:05:16.032521 26120 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ranger-kms
I20260812 19:05:16.032617 26120 mini_ranger_kms.cc:227] Using postgres at 127.25.130.20:38741
2026-08-12 19:05:16,570 [I] DB FLAVOR :POSTGRES
2026-08-12 19:05:16,570 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 19:05:16,570 [I] Checking connection
2026-08-12 19:05:16,570 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:38741/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 19:05:16,855 [I] connection success
2026-08-12 19:05:16,855 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 19:05:16,855 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 19:05:16,855 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:38741/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 19:05:17,104 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 19:05:17,104 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 19:05:17,104 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 19:05:17,104 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:38741/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 19:05:17,369 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 19:05:17.861996 26120 mini_ranger_kms.cc:310] Created kms service
I20260812 19:05:18.044731 26120 mini_ranger_kms.cc:326] Created kudu user
I20260812 19:05:18.092454 26120 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 19:05:18 dist-test-slave-49h3 krb5kdc[6869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561515, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.25.130.20@KRBTEST.COM
I20260812 19:05:18.507973 26120 mini_ranger_kms.cc:384] Added ranger policy
I20260812 19:05:18.508138 26120 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ranger-kms:/tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 19:05:18.508159 26120 mini_ranger_kms.cc:240] Using host: 127.25.130.20
I20260812 19:05:18.512846 26120 mini_ranger_kms.cc:276] Ranger KMS PID: 7343
I20260812 19:05:18.513010 26120 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.25.130.20:45765
Aug 12, 2026 7:05:19 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 7:05:19 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 7:05:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 7:05:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 7:05:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 7:05:20 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260812 19:05:20.820876 26120 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.596s user 0.011s sys 0.493s
I20260812 19:05:20.821012 26120 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 19:05:20.821143 26120 mini_ranger_kms.cc:401] 127.25.130.20:45765/kms/v1/keys
Aug 12 19:05:20 dist-test-slave-49h3 krb5kdc[6869](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786561520, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.130.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 19:05:21 dist-test-slave-49h3 krb5kdc[6869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561520, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.130.20@KRBTEST.COM for HTTP/127.25.130.20@KRBTEST.COM
Aug 12 19:05:21 dist-test-slave-49h3 krb5kdc[6869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561520, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.130.20@KRBTEST.COM for HTTP/127.25.130.20@KRBTEST.COM
Aug 12 19:05:24 dist-test-slave-49h3 krb5kdc[6869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:05:24.319998 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:37413
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:37047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:37413
--ranger_config_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.130.20:45765/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:05:24.453860 7405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:05:24.454149 7405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:05:24.454188 7405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:05:24.458359 7405 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:05:24.458431 7405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:05:24.458451 7405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:05:24.458467 7405 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:05:24.458483 7405 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:05:24.464133 7405 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:37047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/master-0/wal
--ranger_kms_url=127.25.130.20:45765/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:37413
--ranger_config_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:37413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:05:24.465233 7405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:05:24.466326 7405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:05:24.475776 7405 server_base.cc:1034] running on GCE node
W20260812 19:05:24.475708 7412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:05:24.475723 7414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:05:24.475757 7411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:05:24.476673 7405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:05:24.477953 7405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:05:24.479161 7405 hybrid_clock.cc:648] HybridClock initialized: now 1786561524479139 us; error 36 us; skew 500 ppm
Aug 12 19:05:24 dist-test-slave-49h3 krb5kdc[6869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561524, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:05:24.486800 7405 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:05:24.487793 7405 webserver.cc:460] Webserver started at http://127.25.130.62:44715/ using document root <none> and password file <none>
I20260812 19:05:24.488351 7405 fs_manager.cc:359] Metadata directory not provided
I20260812 19:05:24.488413 7405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:05:24.488613 7405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 19:05:24 dist-test-slave-49h3 krb5kdc[6869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561524, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for HTTP/127.25.130.20@KRBTEST.COM
I20260812 19:05:24.548375 7405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "8dcbb9c96fbb49b684c364e8b4c1c488"
format_stamp: "Formatted at 2026-08-12 19:05:24 on dist-test-slave-49h3"
server_key: "c2a83023c66c0760dc5685dc3fba9ed9"
server_key_iv: "2cccb87c91150830715ac7317d29ef96"
server_key_version: "kuduclusterkey@0"
I20260812 19:05:24.549094 7405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "8dcbb9c96fbb49b684c364e8b4c1c488"
format_stamp: "Formatted at 2026-08-12 19:05:24 on dist-test-slave-49h3"
server_key: "c2a83023c66c0760dc5685dc3fba9ed9"
server_key_iv: "2cccb87c91150830715ac7317d29ef96"
server_key_version: "kuduclusterkey@0"
I20260812 19:05:24.553604 7405 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:05:24.573411 7422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:24.575027 7405 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:05:24.575170 7405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "8dcbb9c96fbb49b684c364e8b4c1c488"
format_stamp: "Formatted at 2026-08-12 19:05:24 on dist-test-slave-49h3"
server_key: "c2a83023c66c0760dc5685dc3fba9ed9"
server_key_iv: "2cccb87c91150830715ac7317d29ef96"
server_key_version: "kuduclusterkey@0"
I20260812 19:05:24.575316 7405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:05:24.604295 7405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:05:24.607425 7405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:05:24.607620 7405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:05:24.617733 7405 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:37413
I20260812 19:05:24.617866 7490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:37413 every 8 connection(s)
I20260812 19:05:24.618800 7405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:05:24.620357 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 7405
I20260812 19:05:24.620471 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:05:24.620748 26120 external_mini_cluster.cc:1428] Setting key e8821a09ec462d4af67caff61590b4f3
I20260812 19:05:24.622018 7491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:24.629628 7491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dcbb9c96fbb49b684c364e8b4c1c488: Bootstrap starting.
Aug 12 19:05:24 dist-test-slave-49h3 krb5kdc[6869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:05:24.632474 7491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8dcbb9c96fbb49b684c364e8b4c1c488: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:24.633978 7491 log.cc:824] T 00000000000000000000000000000000 P 8dcbb9c96fbb49b684c364e8b4c1c488: Log is configured to *not* fsync() on all Append() calls
I20260812 19:05:24.636050 7494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:05:24.622297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45816 (local address 127.25.130.62:37413)
0812 19:05:24.622890 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0812 19:05:24.622929 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0812 19:05:24.622969 (+ 40us) server_negotiation.cc:407] Connection header received
0812 19:05:24.623693 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:05:24.623710 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:05:24.624050 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:05:24.624355 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:05:24.625352 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:05:24.626467 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:05:24.627651 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:05:24.628133 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:05:24.628253 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:05:24.631218 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:05:24.631224 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:05:24.631238 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:05:24.631272 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:05:24.633148 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:05:24.633631 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:05:24.633636 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:05:24.633641 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:05:24.633753 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:05:24.634017 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:05:24.634020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:05:24.634021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:05:24.634503 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:05:24.634723 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0812 19:05:24.634986 (+ 263us) server_negotiation.cc:299] Negotiation successful
0812 19:05:24.635253 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":183,"threads_started":1}
I20260812 19:05:24.636200 7491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dcbb9c96fbb49b684c364e8b4c1c488: No bootstrap required, opened a new log
I20260812 19:05:24.639420 7491 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8dcbb9c96fbb49b684c364e8b4c1c488 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dcbb9c96fbb49b684c364e8b4c1c488" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37413 } }
I20260812 19:05:24.639652 7491 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8dcbb9c96fbb49b684c364e8b4c1c488 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:24.639710 7491 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8dcbb9c96fbb49b684c364e8b4c1c488 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dcbb9c96fbb49b684c364e8b4c1c488, State: Initialized, Role: FOLLOWER
I20260812 19:05:24.640125 7491 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8dcbb9c96fbb49b684c364e8b4c1c488 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dcbb9c96fbb49b684c364e8b4c1c488" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37413 } }
I20260812 19:05:24.640257 7491 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8dcbb9c96fbb49b684c364e8b4c1c488 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:05:24.640323 7491 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8dcbb9c96fbb49b684c364e8b4c1c488 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:05:24.640424 7491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8dcbb9c96fbb49b684c364e8b4c1c488 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:24.641592 7491 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8dcbb9c96fbb49b684c364e8b4c1c488 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dcbb9c96fbb49b684c364e8b4c1c488" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37413 } }
I20260812 19:05:24.641927 7491 leader_election.cc:302] T 00000000000000000000000000000000 P 8dcbb9c96fbb49b684c364e8b4c1c488 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8dcbb9c96fbb49b684c364e8b4c1c488; no voters:
I20260812 19:05:24.642267 7491 leader_election.cc:288] T 00000000000000000000000000000000 P 8dcbb9c96fbb49b684c364e8b4c1c488 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:05:24.642375 7497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8dcbb9c96fbb49b684c364e8b4c1c488 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:05:24.642704 7497 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8dcbb9c96fbb49b684c364e8b4c1c488 [term 1 LEADER]: Becoming Leader. State: Replica: 8dcbb9c96fbb49b684c364e8b4c1c488, State: Running, Role: LEADER
I20260812 19:05:24.643020 7497 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8dcbb9c96fbb49b684c364e8b4c1c488 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dcbb9c96fbb49b684c364e8b4c1c488" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37413 } }
I20260812 19:05:24.643577 7491 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8dcbb9c96fbb49b684c364e8b4c1c488 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:05:24.644779 7498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dcbb9c96fbb49b684c364e8b4c1c488 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8dcbb9c96fbb49b684c364e8b4c1c488" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dcbb9c96fbb49b684c364e8b4c1c488" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37413 } } }
I20260812 19:05:24.644910 7498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dcbb9c96fbb49b684c364e8b4c1c488 [sys.catalog]: This master's current role is: LEADER
I20260812 19:05:24.645342 7499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dcbb9c96fbb49b684c364e8b4c1c488 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8dcbb9c96fbb49b684c364e8b4c1c488. Latest consensus state: current_term: 1 leader_uuid: "8dcbb9c96fbb49b684c364e8b4c1c488" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dcbb9c96fbb49b684c364e8b4c1c488" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37413 } } }
I20260812 19:05:24.645414 7506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:05:24.645529 7499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dcbb9c96fbb49b684c364e8b4c1c488 [sys.catalog]: This master's current role is: LEADER
I20260812 19:05:24.648787 7506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:05:24.649434 7491 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 19:05:24.660550 7506 catalog_manager.cc:1347] Generated new cluster ID: 8f83e3171bce4f42ac735e57f709b520
I20260812 19:05:24.660655 7506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:05:24.674942 7506 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:05:24.675959 7506 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:05:24.683202 7506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8dcbb9c96fbb49b684c364e8b4c1c488: Generated new TSK 0
I20260812 19:05:24.683929 7506 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
I20260812 19:05:25.399852 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.130.20:45765/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:0
--local_ip_for_outbound_sockets=127.25.130.1
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:37413
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:37047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:05:25.562642 7542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:05:25.562937 7542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:05:25.562979 7542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:05:25.567250 7542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:05:25.567329 7542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:05:25.567390 7542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:05:25.573604 7542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:37047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.25.130.20:45765/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:37413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:05:25.575103 7542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:05:25.576443 7542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:05:25.588842 7551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:05:25.588773 7552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:05:25.589221 7555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:05:25.589512 7542 server_base.cc:1034] running on GCE node
I20260812 19:05:25.589934 7542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:05:25.590605 7542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:05:25.591802 7542 hybrid_clock.cc:648] HybridClock initialized: now 1786561525591774 us; error 39 us; skew 500 ppm
Aug 12 19:05:25 dist-test-slave-49h3 krb5kdc[6869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561525, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:05:25.594956 7542 init.cc:377] Logged in from keytab as kudu/127.25.130.1@KRBTEST.COM (short username kudu)
I20260812 19:05:25.595959 7542 webserver.cc:460] Webserver started at http://127.25.130.1:34425/ using document root <none> and password file <none>
I20260812 19:05:25.596525 7542 fs_manager.cc:359] Metadata directory not provided
I20260812 19:05:25.596606 7542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:05:25.596851 7542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 19:05:25 dist-test-slave-49h3 krb5kdc[6869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561525, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for HTTP/127.25.130.20@KRBTEST.COM
I20260812 19:05:25.614780 7542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-0/data/instance:
uuid: "2b04acc7f41f4c289a25843a0cef10d6"
format_stamp: "Formatted at 2026-08-12 19:05:25 on dist-test-slave-49h3"
server_key: "ebdd498e239c9c27735df86e2b5dc980"
server_key_iv: "66865926cb1dbb20c8cc3beac0bc9de2"
server_key_version: "kuduclusterkey@0"
I20260812 19:05:25.615990 7542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance:
uuid: "2b04acc7f41f4c289a25843a0cef10d6"
format_stamp: "Formatted at 2026-08-12 19:05:25 on dist-test-slave-49h3"
server_key: "ebdd498e239c9c27735df86e2b5dc980"
server_key_iv: "66865926cb1dbb20c8cc3beac0bc9de2"
server_key_version: "kuduclusterkey@0"
I20260812 19:05:25.627131 7542 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.004s sys 0.000s
I20260812 19:05:25.645773 7562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:25.647208 7542 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 19:05:25.647346 7542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "2b04acc7f41f4c289a25843a0cef10d6"
format_stamp: "Formatted at 2026-08-12 19:05:25 on dist-test-slave-49h3"
server_key: "ebdd498e239c9c27735df86e2b5dc980"
server_key_iv: "66865926cb1dbb20c8cc3beac0bc9de2"
server_key_version: "kuduclusterkey@0"
I20260812 19:05:25.647462 7542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:05:25.665377 7542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:05:25.669271 7542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:05:25.669561 7542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:05:25.670387 7542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:05:25.671554 7542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:05:25.671679 7542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:25.671823 7542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:05:25.671906 7542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:25.692961 7542 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:36045
I20260812 19:05:25.694358 7542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:05:25.695982 7693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:36045 every 8 connection(s)
I20260812 19:05:25.699893 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 7542
I20260812 19:05:25.700037 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance
I20260812 19:05:25.700357 26120 external_mini_cluster.cc:1428] Setting key c1f763a409b6b60d5977d2440177e3aa
Aug 12 19:05:25 dist-test-slave-49h3 krb5kdc[6869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561525, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.130.2@KRBTEST.COM; defaulting to no policy
I20260812 19:05:25.713899 7696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:05:25.697786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:45389 (local address 127.25.130.62:37413)
0812 19:05:25.699537 (+ 1751us) server_negotiation.cc:206] Beginning negotiation
0812 19:05:25.699542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:05:25.699562 (+ 20us) server_negotiation.cc:407] Connection header received
0812 19:05:25.700388 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:05:25.700392 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:05:25.700457 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:05:25.700575 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:05:25.702882 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:05:25.703726 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:05:25.705489 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:05:25.705831 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:05:25.705931 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:05:25.709072 (+ 3141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:05:25.709076 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:05:25.709079 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:05:25.709111 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:05:25.711126 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:05:25.711819 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:05:25.711823 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:05:25.711825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:05:25.711926 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:05:25.712361 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:05:25.712365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:05:25.712367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:05:25.712625 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:05:25.712730 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 19:05:25.713563 (+ 833us) server_negotiation.cc:299] Negotiation successful
0812 19:05:25.713696 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1663,"thread_start_us":139,"threads_started":1}
I20260812 19:05:25.716125 7698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:05:25.698349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37413 (local address 127.25.130.1:45389)
0812 19:05:25.699025 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0812 19:05:25.699091 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 19:05:25.700130 (+ 1039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:05:25.700775 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:05:25.700785 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:05:25.702006 (+ 1221us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:05:25.702703 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:05:25.702737 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:05:25.703877 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:05:25.703881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:05:25.705304 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:05:25.705316 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:05:25.706136 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:05:25.706140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:05:25.706332 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:05:25.707023 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:05:25.707044 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:05:25.708839 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:05:25.711275 (+ 2436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:05:25.711282 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:05:25.711303 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:05:25.711679 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:05:25.712036 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:05:25.712040 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:05:25.712042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:05:25.712246 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:05:25.712791 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:05:25.712798 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:05:25.713144 (+ 346us) client_negotiation.cc:769] Sending connection context
0812 19:05:25.713371 (+ 227us) client_negotiation.cc:240] Negotiation successful
0812 19:05:25.714910 (+ 1539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":176,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.2@KRBTEST.COM" created.
I20260812 19:05:25.718122 7694 heartbeater.cc:347] Connected to a master server at 127.25.130.62:37413
I20260812 19:05:25.718387 7694 heartbeater.cc:464] Registering TS with master...
I20260812 19:05:25.719038 7694 heartbeater.cc:511] Master 127.25.130.62:37413 requested a full tablet report, sending...
I20260812 19:05:25.721079 7445 ts_manager.cc:194] Registered new tserver with Master: 2b04acc7f41f4c289a25843a0cef10d6 (127.25.130.1:36045)
I20260812 19:05:25.722822 7445 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.1@KRBTEST.COM'} at 127.25.130.1:45389
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
I20260812 19:05:25.747171 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.130.20:45765/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:0
--local_ip_for_outbound_sockets=127.25.130.2
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:37413
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:37047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 19:05:25 dist-test-slave-49h3 krb5kdc[6869](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786561525, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 19:05:25.919059 7702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:05:25.919560 7702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:05:25.919688 7702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:05:25.926040 7702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:05:25.926206 7702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:05:25.926333 7702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:05:25.932940 7702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:37047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.25.130.20:45765/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:37413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:05:25.934144 7702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:05:25.935434 7702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:05:25.947556 7709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:05:25.947604 7713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:05:25.948078 7708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:05:25.948206 7702 server_base.cc:1034] running on GCE node
I20260812 19:05:25.948669 7702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:05:25.949347 7702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:05:25.950515 7702 hybrid_clock.cc:648] HybridClock initialized: now 1786561525950480 us; error 45 us; skew 500 ppm
Aug 12 19:05:25 dist-test-slave-49h3 krb5kdc[6869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561525, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:05:25.954499 7702 init.cc:377] Logged in from keytab as kudu/127.25.130.2@KRBTEST.COM (short username kudu)
I20260812 19:05:25.955864 7702 webserver.cc:460] Webserver started at http://127.25.130.2:34921/ using document root <none> and password file <none>
I20260812 19:05:25.956760 7702 fs_manager.cc:359] Metadata directory not provided
I20260812 19:05:25.956912 7702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:05:25.957229 7702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 19:05:25 dist-test-slave-49h3 krb5kdc[6869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561525, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for HTTP/127.25.130.20@KRBTEST.COM
I20260812 19:05:25.974364 7702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-1/data/instance:
uuid: "ef968b90aef744228766ae1f30c9878b"
format_stamp: "Formatted at 2026-08-12 19:05:25 on dist-test-slave-49h3"
server_key: "2ccdb64c229ddfc7c4cf2472dcc786bc"
server_key_iv: "fe2f16146f2ef43d82e071dc839826ce"
server_key_version: "kuduclusterkey@0"
I20260812 19:05:25.975960 7702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance:
uuid: "ef968b90aef744228766ae1f30c9878b"
format_stamp: "Formatted at 2026-08-12 19:05:25 on dist-test-slave-49h3"
server_key: "2ccdb64c229ddfc7c4cf2472dcc786bc"
server_key_iv: "fe2f16146f2ef43d82e071dc839826ce"
server_key_version: "kuduclusterkey@0"
I20260812 19:05:25.981158 7702 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 19:05:26.008168 7723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:26.010255 7702 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.003s sys 0.001s
I20260812 19:05:26.010435 7702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "ef968b90aef744228766ae1f30c9878b"
format_stamp: "Formatted at 2026-08-12 19:05:25 on dist-test-slave-49h3"
server_key: "2ccdb64c229ddfc7c4cf2472dcc786bc"
server_key_iv: "fe2f16146f2ef43d82e071dc839826ce"
server_key_version: "kuduclusterkey@0"
I20260812 19:05:26.010605 7702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:05:26.025899 7702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:05:26.030020 7702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:05:26.030310 7702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:05:26.031157 7702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:05:26.032366 7702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:05:26.032480 7702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:26.032583 7702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:05:26.032649 7702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:26.049307 7702 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:39995
I20260812 19:05:26.050211 7860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:39995 every 8 connection(s)
I20260812 19:05:26.051023 7702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:05:26.059105 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 7702
I20260812 19:05:26.059232 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance
I20260812 19:05:26.059491 26120 external_mini_cluster.cc:1428] Setting key 06e79c6608b7f5edeee50e58f6edac96
Aug 12 19:05:26 dist-test-slave-49h3 krb5kdc[6869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561525, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:05:26.072330 7696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:05:26.054673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:49467 (local address 127.25.130.62:37413)
0812 19:05:26.054787 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 19:05:26.054791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:05:26.055078 (+ 287us) server_negotiation.cc:407] Connection header received
0812 19:05:26.056054 (+ 976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:05:26.056058 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:05:26.056116 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:05:26.056175 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:05:26.057890 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.058769 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.062630 (+ 3861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.062939 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.063019 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:05:26.067753 (+ 4734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:05:26.067757 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:05:26.067760 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:05:26.067790 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:05:26.069568 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:05:26.070289 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:05:26.070293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:05:26.070296 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:05:26.070393 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:05:26.070845 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:05:26.070849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:05:26.070851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:05:26.071104 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:05:26.071190 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 19:05:26.071993 (+ 803us) server_negotiation.cc:299] Negotiation successful
0812 19:05:26.072141 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 19:05:26.074488 7864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:05:26.053649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37413 (local address 127.25.130.2:49467)
0812 19:05:26.054926 (+ 1277us) negotiation.cc:107] Waiting for socket to connect
0812 19:05:26.054968 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 19:05:26.055788 (+ 820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:05:26.056596 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:05:26.056606 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:05:26.056976 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:05:26.057703 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:05:26.057726 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.059986 (+ 2260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.059989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:05:26.062478 (+ 2489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:05:26.062491 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.063274 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.063278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:05:26.063511 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:05:26.064271 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:05:26.064294 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:05:26.067521 (+ 3227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:05:26.069731 (+ 2210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:05:26.069738 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:05:26.069758 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:05:26.070156 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:05:26.070506 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:05:26.070509 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:05:26.070512 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:05:26.070727 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:05:26.071268 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:05:26.071275 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:05:26.071677 (+ 402us) client_negotiation.cc:769] Sending connection context
0812 19:05:26.072881 (+ 1204us) client_negotiation.cc:240] Negotiation successful
0812 19:05:26.073283 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":973,"thread_start_us":147,"threads_started":1}
WARNING: no policy specified for kudu/127.25.130.3@KRBTEST.COM; defaulting to no policy
I20260812 19:05:26.076257 7861 heartbeater.cc:347] Connected to a master server at 127.25.130.62:37413
I20260812 19:05:26.076551 7861 heartbeater.cc:464] Registering TS with master...
I20260812 19:05:26.077193 7861 heartbeater.cc:511] Master 127.25.130.62:37413 requested a full tablet report, sending...
I20260812 19:05:26.078658 7445 ts_manager.cc:194] Registered new tserver with Master: ef968b90aef744228766ae1f30c9878b (127.25.130.2:39995)
I20260812 19:05:26.079736 7445 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.2@KRBTEST.COM'} at 127.25.130.2:49467
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
I20260812 19:05:26.109210 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.130.20:45765/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:0
--local_ip_for_outbound_sockets=127.25.130.3
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:37413
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:37047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:05:26.365360 7867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:05:26.365828 7867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:05:26.365940 7867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
Aug 12 19:05:26 dist-test-slave-49h3 krb5kdc[6869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561525, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for HTTP/127.25.130.20@KRBTEST.COM
W20260812 19:05:26.371865 7867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:05:26.372004 7867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:05:26.372125 7867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:05:26.380479 7867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:37047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.25.130.20:45765/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:37413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:05:26.382855 7867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:05:26.384375 7867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:05:26.408303 7873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:05:26.408699 7875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:05:26.408928 7872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:05:26.410420 7867 server_base.cc:1034] running on GCE node
I20260812 19:05:26.411180 7867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:05:26.412081 7867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:05:26.413316 7867 hybrid_clock.cc:648] HybridClock initialized: now 1786561526413285 us; error 54 us; skew 500 ppm
Aug 12 19:05:26 dist-test-slave-49h3 krb5kdc[6869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561526, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:05:26.417713 7867 init.cc:377] Logged in from keytab as kudu/127.25.130.3@KRBTEST.COM (short username kudu)
I20260812 19:05:26.419018 7867 webserver.cc:460] Webserver started at http://127.25.130.3:37703/ using document root <none> and password file <none>
I20260812 19:05:26.419835 7867 fs_manager.cc:359] Metadata directory not provided
I20260812 19:05:26.419994 7867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:05:26.420329 7867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 19:05:26 dist-test-slave-49h3 krb5kdc[6869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561526, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for HTTP/127.25.130.20@KRBTEST.COM
I20260812 19:05:26.448173 7867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-2/data/instance:
uuid: "7317b99f036c48068aed6530d890a0f9"
format_stamp: "Formatted at 2026-08-12 19:05:26 on dist-test-slave-49h3"
server_key: "6e9f2343c92227f7af901ddb39ddb978"
server_key_iv: "a78b60377190be1934a2c5a0d5db4169"
server_key_version: "kuduclusterkey@0"
I20260812 19:05:26.449291 7867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance:
uuid: "7317b99f036c48068aed6530d890a0f9"
format_stamp: "Formatted at 2026-08-12 19:05:26 on dist-test-slave-49h3"
server_key: "6e9f2343c92227f7af901ddb39ddb978"
server_key_iv: "a78b60377190be1934a2c5a0d5db4169"
server_key_version: "kuduclusterkey@0"
I20260812 19:05:26.455649 7867 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.001s
I20260812 19:05:26.475404 7884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:26.477245 7867 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 19:05:26.477497 7867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "7317b99f036c48068aed6530d890a0f9"
format_stamp: "Formatted at 2026-08-12 19:05:26 on dist-test-slave-49h3"
server_key: "6e9f2343c92227f7af901ddb39ddb978"
server_key_iv: "a78b60377190be1934a2c5a0d5db4169"
server_key_version: "kuduclusterkey@0"
I20260812 19:05:26.477741 7867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:05:26.505275 7867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:05:26.511294 7867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:05:26.511576 7867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:05:26.512373 7867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:05:26.513586 7867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:05:26.513702 7867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:26.513795 7867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:05:26.513851 7867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:26.528314 7867 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:44489
I20260812 19:05:26.529709 7867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:05:26.537832 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 7867
I20260812 19:05:26.537987 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance
I20260812 19:05:26.538311 26120 external_mini_cluster.cc:1428] Setting key 44b50969e3080ddd85ba37f113f79352
I20260812 19:05:26.541674 8021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:44489 every 8 connection(s)
Aug 12 19:05:26 dist-test-slave-49h3 krb5kdc[6869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561526, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:05:26.559237 7696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:05:26.537382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:33961 (local address 127.25.130.62:37413)
0812 19:05:26.538804 (+ 1422us) server_negotiation.cc:206] Beginning negotiation
0812 19:05:26.538811 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 19:05:26.542626 (+ 3815us) server_negotiation.cc:407] Connection header received
0812 19:05:26.544007 (+ 1381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:05:26.544011 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:05:26.544070 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:05:26.544121 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:05:26.546383 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.547218 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.549004 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.549342 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.549417 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:05:26.554886 (+ 5469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:05:26.554890 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:05:26.554893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:05:26.554924 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:05:26.556616 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:05:26.557342 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:05:26.557346 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:05:26.557348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:05:26.557437 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:05:26.557904 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:05:26.557907 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:05:26.557909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:05:26.558143 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:05:26.558217 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 19:05:26.558942 (+ 725us) server_negotiation.cc:299] Negotiation successful
0812 19:05:26.559075 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1288}
I20260812 19:05:26.561245 8025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:05:26.536134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37413 (local address 127.25.130.3:33961)
0812 19:05:26.542443 (+ 6309us) negotiation.cc:107] Waiting for socket to connect
0812 19:05:26.542521 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0812 19:05:26.543684 (+ 1163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:05:26.544967 (+ 1283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:05:26.544978 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:05:26.545416 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:05:26.546204 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:05:26.546225 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.547420 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.547423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:05:26.548848 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:05:26.548862 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.549686 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.549690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:05:26.549909 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:05:26.550874 (+ 965us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:05:26.550900 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:05:26.554671 (+ 3771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:05:26.556820 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:05:26.556828 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:05:26.556850 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:05:26.557225 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:05:26.557574 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:05:26.557577 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:05:26.557579 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:05:26.557798 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:05:26.558282 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:05:26.558289 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:05:26.558659 (+ 370us) client_negotiation.cc:769] Sending connection context
0812 19:05:26.559733 (+ 1074us) client_negotiation.cc:240] Negotiation successful
0812 19:05:26.560111 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6023,"thread_start_us":165,"threads_started":1}
I20260812 19:05:26.562949 8023 heartbeater.cc:347] Connected to a master server at 127.25.130.62:37413
I20260812 19:05:26.563230 8023 heartbeater.cc:464] Registering TS with master...
I20260812 19:05:26.563862 8023 heartbeater.cc:511] Master 127.25.130.62:37413 requested a full tablet report, sending...
I20260812 19:05:26.566843 7445 ts_manager.cc:194] Registered new tserver with Master: 7317b99f036c48068aed6530d890a0f9 (127.25.130.3:44489)
I20260812 19:05:26.567631 7445 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.3@KRBTEST.COM'} at 127.25.130.3:33961
I20260812 19:05:26.578593 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:05:26.591248 7696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:05:26.581113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34172 (local address 127.25.130.62:37413)
0812 19:05:26.581238 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 19:05:26.581242 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:05:26.581582 (+ 340us) server_negotiation.cc:407] Connection header received
0812 19:05:26.581730 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:05:26.581733 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:05:26.581788 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:05:26.581878 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:05:26.582837 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.583737 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.584902 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.585401 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.585601 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:05:26.587352 (+ 1751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:05:26.587356 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:05:26.587359 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:05:26.587392 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:05:26.589121 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:05:26.589744 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:05:26.589748 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:05:26.589750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:05:26.589845 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:05:26.590156 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:05:26.590159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:05:26.590161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:05:26.590398 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:05:26.590503 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 19:05:26.590928 (+ 425us) server_negotiation.cc:299] Negotiation successful
0812 19:05:26.591065 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 19:05:26.595489 7445 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34172:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:05:26.599965 7445 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:05:26.618062 8036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:05:26.610175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:39995 (local address 127.0.0.1:43920)
0812 19:05:26.610630 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0812 19:05:26.610686 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 19:05:26.610817 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:05:26.613073 (+ 2256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:05:26.613079 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:05:26.613105 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:05:26.613258 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:05:26.613265 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.614877 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.614880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:05:26.616259 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:05:26.616271 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.617474 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.617478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:05:26.617622 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:05:26.617685 (+ 63us) client_negotiation.cc:769] Sending connection context
0812 19:05:26.617775 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 19:05:26.617895 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":146,"threads_started":1}
I20260812 19:05:26.620698 8037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:05:26.611610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:36045 (local address 127.0.0.1:38318)
0812 19:05:26.613577 (+ 1967us) negotiation.cc:107] Waiting for socket to connect
0812 19:05:26.613590 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:05:26.613750 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:05:26.614388 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:05:26.614391 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:05:26.614408 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:05:26.614500 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:05:26.614505 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.616793 (+ 2288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.616796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:05:26.618060 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:05:26.618072 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.620263 (+ 2191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.620267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:05:26.620356 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:05:26.620381 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 19:05:26.620458 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 19:05:26.620551 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1898,"thread_start_us":181,"threads_started":1}
I20260812 19:05:26.621299 8040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:05:26.611756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38318 (local address 127.25.130.1:36045)
0812 19:05:26.612119 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0812 19:05:26.612126 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 19:05:26.613682 (+ 1556us) server_negotiation.cc:407] Connection header received
0812 19:05:26.613884 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:05:26.613890 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:05:26.614092 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:05:26.614214 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:05:26.614641 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.616645 (+ 2004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.618206 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.619899 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.619987 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:05:26.620153 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 19:05:26.621041 (+ 888us) server_negotiation.cc:299] Negotiation successful
0812 19:05:26.621156 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":89,"threads_started":1}
I20260812 19:05:26.624694 7598 tablet_service.cc:1467] Processing CreateTablet for tablet 3d62819dc7d64e45870511ba49657603 (DEFAULT_TABLE table=test-table [id=04ddfb56381c4442b2dd1fae15d8ce24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:05:26.627166 7598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d62819dc7d64e45870511ba49657603. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:26.631557 8041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:05:26.612014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43920 (local address 127.25.130.2:39995)
0812 19:05:26.612307 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0812 19:05:26.612313 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:05:26.612338 (+ 25us) server_negotiation.cc:407] Connection header received
0812 19:05:26.612420 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:05:26.612425 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:05:26.612895 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:05:26.613414 (+ 519us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:05:26.613604 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.614745 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.616420 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.617320 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.631012 (+ 13692us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:05:26.631207 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0812 19:05:26.631271 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 19:05:26.631385 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":115,"threads_started":1}
I20260812 19:05:26.634899 7783 tablet_service.cc:1467] Processing CreateTablet for tablet 3d62819dc7d64e45870511ba49657603 (DEFAULT_TABLE table=test-table [id=04ddfb56381c4442b2dd1fae15d8ce24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:05:26.636088 7783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d62819dc7d64e45870511ba49657603. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:26.648295 8043 tablet_bootstrap.cc:492] T 3d62819dc7d64e45870511ba49657603 P ef968b90aef744228766ae1f30c9878b: Bootstrap starting.
I20260812 19:05:26.649785 8039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:05:26.611065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:44489 (local address 127.0.0.1:43502)
0812 19:05:26.613256 (+ 2191us) negotiation.cc:107] Waiting for socket to connect
0812 19:05:26.613271 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:05:26.613361 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:05:26.639999 (+ 26638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:05:26.640003 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:05:26.640029 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:05:26.640161 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:05:26.640168 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.642000 (+ 1832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.642004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:05:26.643257 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:05:26.643274 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.649248 (+ 5974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.649253 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:05:26.649369 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:05:26.649406 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 19:05:26.649503 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 19:05:26.649616 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2124,"thread_start_us":97,"threads_started":1}
I20260812 19:05:26.650925 8038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:05:26.611249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43502 (local address 127.25.130.3:44489)
0812 19:05:26.639410 (+ 28161us) server_negotiation.cc:206] Beginning negotiation
0812 19:05:26.639435 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0812 19:05:26.639465 (+ 30us) server_negotiation.cc:407] Connection header received
0812 19:05:26.639569 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:05:26.639576 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:05:26.639804 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:05:26.640331 (+ 527us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:05:26.640493 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.641849 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.648084 (+ 6235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.649076 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.650309 (+ 1233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:05:26.650580 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0812 19:05:26.650645 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 19:05:26.650756 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27917,"thread_start_us":119,"threads_started":1}
I20260812 19:05:26.651721 8042 tablet_bootstrap.cc:492] T 3d62819dc7d64e45870511ba49657603 P 2b04acc7f41f4c289a25843a0cef10d6: Bootstrap starting.
I20260812 19:05:26.654354 8042 tablet_bootstrap.cc:654] T 3d62819dc7d64e45870511ba49657603 P 2b04acc7f41f4c289a25843a0cef10d6: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:26.655068 7942 tablet_service.cc:1467] Processing CreateTablet for tablet 3d62819dc7d64e45870511ba49657603 (DEFAULT_TABLE table=test-table [id=04ddfb56381c4442b2dd1fae15d8ce24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:05:26.656351 7942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d62819dc7d64e45870511ba49657603. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:26.656638 8042 log.cc:824] T 3d62819dc7d64e45870511ba49657603 P 2b04acc7f41f4c289a25843a0cef10d6: Log is configured to *not* fsync() on all Append() calls
I20260812 19:05:26.659181 8042 tablet_bootstrap.cc:492] T 3d62819dc7d64e45870511ba49657603 P 2b04acc7f41f4c289a25843a0cef10d6: No bootstrap required, opened a new log
I20260812 19:05:26.659402 8042 ts_tablet_manager.cc:1397] T 3d62819dc7d64e45870511ba49657603 P 2b04acc7f41f4c289a25843a0cef10d6: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 19:05:26.664567 8042 raft_consensus.cc:348] T 3d62819dc7d64e45870511ba49657603 P 2b04acc7f41f4c289a25843a0cef10d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef968b90aef744228766ae1f30c9878b" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 39995 } } peers { permanent_uuid: "7317b99f036c48068aed6530d890a0f9" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 44489 } } peers { permanent_uuid: "2b04acc7f41f4c289a25843a0cef10d6" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36045 } }
I20260812 19:05:26.665061 8042 raft_consensus.cc:374] T 3d62819dc7d64e45870511ba49657603 P 2b04acc7f41f4c289a25843a0cef10d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:26.665185 8042 raft_consensus.cc:729] T 3d62819dc7d64e45870511ba49657603 P 2b04acc7f41f4c289a25843a0cef10d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b04acc7f41f4c289a25843a0cef10d6, State: Initialized, Role: FOLLOWER
I20260812 19:05:26.665933 8043 tablet_bootstrap.cc:654] T 3d62819dc7d64e45870511ba49657603 P ef968b90aef744228766ae1f30c9878b: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:26.667065 8043 log.cc:824] T 3d62819dc7d64e45870511ba49657603 P ef968b90aef744228766ae1f30c9878b: Log is configured to *not* fsync() on all Append() calls
I20260812 19:05:26.665861 8042 consensus_queue.cc:260] T 3d62819dc7d64e45870511ba49657603 P 2b04acc7f41f4c289a25843a0cef10d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef968b90aef744228766ae1f30c9878b" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 39995 } } peers { permanent_uuid: "7317b99f036c48068aed6530d890a0f9" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 44489 } } peers { permanent_uuid: "2b04acc7f41f4c289a25843a0cef10d6" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36045 } }
I20260812 19:05:26.668326 7694 heartbeater.cc:503] Master 127.25.130.62:37413 was elected leader, sending a full tablet report...
I20260812 19:05:26.668810 8042 ts_tablet_manager.cc:1428] T 3d62819dc7d64e45870511ba49657603 P 2b04acc7f41f4c289a25843a0cef10d6: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260812 19:05:26.672820 8046 tablet_bootstrap.cc:492] T 3d62819dc7d64e45870511ba49657603 P 7317b99f036c48068aed6530d890a0f9: Bootstrap starting.
I20260812 19:05:26.675657 8043 tablet_bootstrap.cc:492] T 3d62819dc7d64e45870511ba49657603 P ef968b90aef744228766ae1f30c9878b: No bootstrap required, opened a new log
I20260812 19:05:26.675987 8043 ts_tablet_manager.cc:1397] T 3d62819dc7d64e45870511ba49657603 P ef968b90aef744228766ae1f30c9878b: Time spent bootstrapping tablet: real 0.028s user 0.007s sys 0.000s
I20260812 19:05:26.677884 8046 tablet_bootstrap.cc:654] T 3d62819dc7d64e45870511ba49657603 P 7317b99f036c48068aed6530d890a0f9: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:26.680154 8046 log.cc:824] T 3d62819dc7d64e45870511ba49657603 P 7317b99f036c48068aed6530d890a0f9: Log is configured to *not* fsync() on all Append() calls
I20260812 19:05:26.688931 8043 raft_consensus.cc:348] T 3d62819dc7d64e45870511ba49657603 P ef968b90aef744228766ae1f30c9878b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef968b90aef744228766ae1f30c9878b" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 39995 } } peers { permanent_uuid: "7317b99f036c48068aed6530d890a0f9" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 44489 } } peers { permanent_uuid: "2b04acc7f41f4c289a25843a0cef10d6" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36045 } }
I20260812 19:05:26.689185 8046 tablet_bootstrap.cc:492] T 3d62819dc7d64e45870511ba49657603 P 7317b99f036c48068aed6530d890a0f9: No bootstrap required, opened a new log
I20260812 19:05:26.689375 8043 raft_consensus.cc:374] T 3d62819dc7d64e45870511ba49657603 P ef968b90aef744228766ae1f30c9878b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:26.689527 8043 raft_consensus.cc:729] T 3d62819dc7d64e45870511ba49657603 P ef968b90aef744228766ae1f30c9878b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef968b90aef744228766ae1f30c9878b, State: Initialized, Role: FOLLOWER
I20260812 19:05:26.689659 8046 ts_tablet_manager.cc:1397] T 3d62819dc7d64e45870511ba49657603 P 7317b99f036c48068aed6530d890a0f9: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.000s
I20260812 19:05:26.690192 8043 consensus_queue.cc:260] T 3d62819dc7d64e45870511ba49657603 P ef968b90aef744228766ae1f30c9878b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef968b90aef744228766ae1f30c9878b" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 39995 } } peers { permanent_uuid: "7317b99f036c48068aed6530d890a0f9" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 44489 } } peers { permanent_uuid: "2b04acc7f41f4c289a25843a0cef10d6" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36045 } }
I20260812 19:05:26.691694 8043 ts_tablet_manager.cc:1428] T 3d62819dc7d64e45870511ba49657603 P ef968b90aef744228766ae1f30c9878b: Time spent starting tablet: real 0.016s user 0.007s sys 0.000s
I20260812 19:05:26.692145 7861 heartbeater.cc:503] Master 127.25.130.62:37413 was elected leader, sending a full tablet report...
I20260812 19:05:26.693786 8046 raft_consensus.cc:348] T 3d62819dc7d64e45870511ba49657603 P 7317b99f036c48068aed6530d890a0f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef968b90aef744228766ae1f30c9878b" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 39995 } } peers { permanent_uuid: "7317b99f036c48068aed6530d890a0f9" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 44489 } } peers { permanent_uuid: "2b04acc7f41f4c289a25843a0cef10d6" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36045 } }
I20260812 19:05:26.694360 8046 raft_consensus.cc:374] T 3d62819dc7d64e45870511ba49657603 P 7317b99f036c48068aed6530d890a0f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:26.694479 8046 raft_consensus.cc:729] T 3d62819dc7d64e45870511ba49657603 P 7317b99f036c48068aed6530d890a0f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7317b99f036c48068aed6530d890a0f9, State: Initialized, Role: FOLLOWER
I20260812 19:05:26.695081 8046 consensus_queue.cc:260] T 3d62819dc7d64e45870511ba49657603 P 7317b99f036c48068aed6530d890a0f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef968b90aef744228766ae1f30c9878b" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 39995 } } peers { permanent_uuid: "7317b99f036c48068aed6530d890a0f9" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 44489 } } peers { permanent_uuid: "2b04acc7f41f4c289a25843a0cef10d6" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36045 } }
W20260812 19:05:26.697575 7695 tablet.cc:2369] T 3d62819dc7d64e45870511ba49657603 P 2b04acc7f41f4c289a25843a0cef10d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:05:26.698218 8046 ts_tablet_manager.cc:1428] T 3d62819dc7d64e45870511ba49657603 P 7317b99f036c48068aed6530d890a0f9: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260812 19:05:26.698634 8023 heartbeater.cc:503] Master 127.25.130.62:37413 was elected leader, sending a full tablet report...
Aug 12 19:05:26 dist-test-slave-49h3 krb5kdc[6869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561525, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for HTTP/127.25.130.20@KRBTEST.COM
W20260812 19:05:26.781363 8024 tablet.cc:2369] T 3d62819dc7d64e45870511ba49657603 P 7317b99f036c48068aed6530d890a0f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 19:05:26.803740 7863 tablet.cc:2369] T 3d62819dc7d64e45870511ba49657603 P ef968b90aef744228766ae1f30c9878b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:05:26.805316 8048 raft_consensus.cc:482] T 3d62819dc7d64e45870511ba49657603 P 2b04acc7f41f4c289a25843a0cef10d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:26.805656 8048 raft_consensus.cc:504] T 3d62819dc7d64e45870511ba49657603 P 2b04acc7f41f4c289a25843a0cef10d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef968b90aef744228766ae1f30c9878b" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 39995 } } peers { permanent_uuid: "7317b99f036c48068aed6530d890a0f9" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 44489 } } peers { permanent_uuid: "2b04acc7f41f4c289a25843a0cef10d6" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36045 } }
I20260812 19:05:26.807049 8048 leader_election.cc:288] T 3d62819dc7d64e45870511ba49657603 P 2b04acc7f41f4c289a25843a0cef10d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef968b90aef744228766ae1f30c9878b (127.25.130.2:39995), 7317b99f036c48068aed6530d890a0f9 (127.25.130.3:44489)
I20260812 19:05:26.819064 8041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:05:26.808247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:41227 (local address 127.25.130.2:39995)
0812 19:05:26.808538 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0812 19:05:26.808543 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:05:26.813917 (+ 5374us) server_negotiation.cc:407] Connection header received
0812 19:05:26.814203 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:05:26.814208 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:05:26.814273 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:05:26.814328 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:05:26.814788 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.815559 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.817407 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.818149 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.818255 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:05:26.818314 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 19:05:26.818811 (+ 497us) server_negotiation.cc:299] Negotiation successful
0812 19:05:26.818907 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195}
I20260812 19:05:26.819804 8059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:05:26.807433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:39995 (local address 127.25.130.1:41227)
0812 19:05:26.813774 (+ 6341us) negotiation.cc:107] Waiting for socket to connect
0812 19:05:26.813795 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 19:05:26.814034 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:05:26.814499 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:05:26.814502 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:05:26.814526 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:05:26.814638 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:05:26.814645 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.815729 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.815733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:05:26.817251 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:05:26.817265 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.818516 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.818519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:05:26.818636 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:05:26.818680 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 19:05:26.819527 (+ 847us) client_negotiation.cc:240] Negotiation successful
0812 19:05:26.819616 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6217,"mutex_wait_us":1,"thread_start_us":133,"threads_started":1}
I20260812 19:05:26.820708 7807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d62819dc7d64e45870511ba49657603" candidate_uuid: "2b04acc7f41f4c289a25843a0cef10d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef968b90aef744228766ae1f30c9878b" is_pre_election: true
I20260812 19:05:26.821077 7807 raft_consensus.cc:2454] T 3d62819dc7d64e45870511ba49657603 P ef968b90aef744228766ae1f30c9878b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b04acc7f41f4c289a25843a0cef10d6 in term 0.
I20260812 19:05:26.821867 7563 leader_election.cc:302] T 3d62819dc7d64e45870511ba49657603 P 2b04acc7f41f4c289a25843a0cef10d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b04acc7f41f4c289a25843a0cef10d6, ef968b90aef744228766ae1f30c9878b; no voters:
I20260812 19:05:26.823189 8048 raft_consensus.cc:2781] T 3d62819dc7d64e45870511ba49657603 P 2b04acc7f41f4c289a25843a0cef10d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:05:26.823323 8048 raft_consensus.cc:482] T 3d62819dc7d64e45870511ba49657603 P 2b04acc7f41f4c289a25843a0cef10d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:05:26.823369 8048 raft_consensus.cc:3037] T 3d62819dc7d64e45870511ba49657603 P 2b04acc7f41f4c289a25843a0cef10d6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:26.824921 8048 raft_consensus.cc:504] T 3d62819dc7d64e45870511ba49657603 P 2b04acc7f41f4c289a25843a0cef10d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef968b90aef744228766ae1f30c9878b" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 39995 } } peers { permanent_uuid: "7317b99f036c48068aed6530d890a0f9" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 44489 } } peers { permanent_uuid: "2b04acc7f41f4c289a25843a0cef10d6" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36045 } }
I20260812 19:05:26.825400 8048 leader_election.cc:288] T 3d62819dc7d64e45870511ba49657603 P 2b04acc7f41f4c289a25843a0cef10d6 [CANDIDATE]: Term 1 election: Requested vote from peers ef968b90aef744228766ae1f30c9878b (127.25.130.2:39995), 7317b99f036c48068aed6530d890a0f9 (127.25.130.3:44489)
I20260812 19:05:26.826259 7807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d62819dc7d64e45870511ba49657603" candidate_uuid: "2b04acc7f41f4c289a25843a0cef10d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef968b90aef744228766ae1f30c9878b"
I20260812 19:05:26.826421 7807 raft_consensus.cc:3037] T 3d62819dc7d64e45870511ba49657603 P ef968b90aef744228766ae1f30c9878b [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:26.826565 8038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:05:26.808370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:48245 (local address 127.25.130.3:44489)
0812 19:05:26.808635 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0812 19:05:26.808639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:05:26.816530 (+ 7891us) server_negotiation.cc:407] Connection header received
0812 19:05:26.816739 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:05:26.816743 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:05:26.816798 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:05:26.816845 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:05:26.817421 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.818145 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.822260 (+ 4115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.822890 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.823032 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:05:26.823091 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 19:05:26.826311 (+ 3220us) server_negotiation.cc:299] Negotiation successful
0812 19:05:26.826424 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189}
I20260812 19:05:26.827209 8057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:05:26.807853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:44489 (local address 127.25.130.1:48245)
0812 19:05:26.816353 (+ 8500us) negotiation.cc:107] Waiting for socket to connect
0812 19:05:26.816370 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 19:05:26.816613 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:05:26.816981 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:05:26.816985 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:05:26.817001 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:05:26.817271 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:05:26.817284 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.818342 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.818345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:05:26.819622 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:05:26.819634 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.825952 (+ 6318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.825957 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:05:26.826111 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:05:26.826143 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 19:05:26.826763 (+ 620us) client_negotiation.cc:240] Negotiation successful
0812 19:05:26.826864 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8414,"mutex_wait_us":28,"thread_start_us":101,"threads_started":1}
I20260812 19:05:26.828017 7807 raft_consensus.cc:2454] T 3d62819dc7d64e45870511ba49657603 P ef968b90aef744228766ae1f30c9878b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b04acc7f41f4c289a25843a0cef10d6 in term 1.
I20260812 19:05:26.828994 7951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d62819dc7d64e45870511ba49657603" candidate_uuid: "2b04acc7f41f4c289a25843a0cef10d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7317b99f036c48068aed6530d890a0f9" is_pre_election: true
I20260812 19:05:26.829360 7951 raft_consensus.cc:2454] T 3d62819dc7d64e45870511ba49657603 P 7317b99f036c48068aed6530d890a0f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b04acc7f41f4c289a25843a0cef10d6 in term 0.
I20260812 19:05:26.829741 7952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d62819dc7d64e45870511ba49657603" candidate_uuid: "2b04acc7f41f4c289a25843a0cef10d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7317b99f036c48068aed6530d890a0f9"
I20260812 19:05:26.829851 7952 raft_consensus.cc:3037] T 3d62819dc7d64e45870511ba49657603 P 7317b99f036c48068aed6530d890a0f9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:26.831281 7952 raft_consensus.cc:2454] T 3d62819dc7d64e45870511ba49657603 P 7317b99f036c48068aed6530d890a0f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b04acc7f41f4c289a25843a0cef10d6 in term 1.
I20260812 19:05:26.832351 7563 leader_election.cc:302] T 3d62819dc7d64e45870511ba49657603 P 2b04acc7f41f4c289a25843a0cef10d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b04acc7f41f4c289a25843a0cef10d6, 7317b99f036c48068aed6530d890a0f9; no voters:
I20260812 19:05:26.832572 8048 raft_consensus.cc:2781] T 3d62819dc7d64e45870511ba49657603 P 2b04acc7f41f4c289a25843a0cef10d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:05:26.832851 8048 raft_consensus.cc:686] T 3d62819dc7d64e45870511ba49657603 P 2b04acc7f41f4c289a25843a0cef10d6 [term 1 LEADER]: Becoming Leader. State: Replica: 2b04acc7f41f4c289a25843a0cef10d6, State: Running, Role: LEADER
I20260812 19:05:26.833243 8048 consensus_queue.cc:237] T 3d62819dc7d64e45870511ba49657603 P 2b04acc7f41f4c289a25843a0cef10d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef968b90aef744228766ae1f30c9878b" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 39995 } } peers { permanent_uuid: "7317b99f036c48068aed6530d890a0f9" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 44489 } } peers { permanent_uuid: "2b04acc7f41f4c289a25843a0cef10d6" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36045 } }
I20260812 19:05:26.837041 7445 catalog_manager.cc:5495] T 3d62819dc7d64e45870511ba49657603 P 2b04acc7f41f4c289a25843a0cef10d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b04acc7f41f4c289a25843a0cef10d6 (127.25.130.1). New cstate: current_term: 1 leader_uuid: "2b04acc7f41f4c289a25843a0cef10d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef968b90aef744228766ae1f30c9878b" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 39995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7317b99f036c48068aed6530d890a0f9" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 44489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b04acc7f41f4c289a25843a0cef10d6" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36045 } health_report { overall_health: HEALTHY } } }
I20260812 19:05:26.872301 8040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:05:26.866954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38322 (local address 127.25.130.1:36045)
0812 19:05:26.867116 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 19:05:26.867121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:05:26.867141 (+ 20us) server_negotiation.cc:407] Connection header received
0812 19:05:26.867258 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:05:26.867262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:05:26.867337 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:05:26.867474 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:05:26.867826 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.868588 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.869872 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:05:26.870205 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:05:26.870312 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:05:26.870795 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:05:26.871435 (+ 640us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:05:26.871552 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 19:05:26.872051 (+ 499us) server_negotiation.cc:299] Negotiation successful
0812 19:05:26.872122 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 19:05:26.880633 7952 raft_consensus.cc:1261] T 3d62819dc7d64e45870511ba49657603 P 7317b99f036c48068aed6530d890a0f9 [term 1 FOLLOWER]: Refusing update from remote peer 2b04acc7f41f4c289a25843a0cef10d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:05:26.881639 8060 consensus_queue.cc:1035] T 3d62819dc7d64e45870511ba49657603 P 2b04acc7f41f4c289a25843a0cef10d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7317b99f036c48068aed6530d890a0f9" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 44489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:05:26.888360 7807 raft_consensus.cc:1261] T 3d62819dc7d64e45870511ba49657603 P ef968b90aef744228766ae1f30c9878b [term 1 FOLLOWER]: Refusing update from remote peer 2b04acc7f41f4c289a25843a0cef10d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:05:26.889233 8060 consensus_queue.cc:1035] T 3d62819dc7d64e45870511ba49657603 P 2b04acc7f41f4c289a25843a0cef10d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef968b90aef744228766ae1f30c9878b" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 39995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:05:26.904850 8068 mvcc.cc:205] Tried to move back new op lower bound from 7317756014093819904 to 7317756013913251840. Current Snapshot: MvccSnapshot[applied={T|T < 7317756014093819904}]
I20260812 19:05:26.907109 8065 mvcc.cc:205] Tried to move back new op lower bound from 7317756014093819904 to 7317756013913251840. Current Snapshot: MvccSnapshot[applied={T|T < 7317756014093819904}]
I20260812 19:05:26.910983 8066 mvcc.cc:205] Tried to move back new op lower bound from 7317756014093819904 to 7317756013913251840. Current Snapshot: MvccSnapshot[applied={T|T < 7317756014093819904}]
I20260812 19:05:26.922679 7445 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:05:26.922888 7445 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:05:26.926218 7445 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8dcbb9c96fbb49b684c364e8b4c1c488: Sending DeleteTablet for 3 replicas of tablet 3d62819dc7d64e45870511ba49657603
I20260812 19:05:26.927938 7783 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d62819dc7d64e45870511ba49657603 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:05:26 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:43920
I20260812 19:05:26.928593 7942 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d62819dc7d64e45870511ba49657603 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:05:26 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:43502
I20260812 19:05:26.929097 7598 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d62819dc7d64e45870511ba49657603 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:05:26 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:38318
I20260812 19:05:26.930657 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 7542
I20260812 19:05:26.940510 8075 tablet_replica.cc:330] T 3d62819dc7d64e45870511ba49657603 P 7317b99f036c48068aed6530d890a0f9: stopping tablet replica
I20260812 19:05:26.946900 8074 tablet_replica.cc:330] T 3d62819dc7d64e45870511ba49657603 P ef968b90aef744228766ae1f30c9878b: stopping tablet replica
I20260812 19:05:26.947492 8075 raft_consensus.cc:2229] T 3d62819dc7d64e45870511ba49657603 P 7317b99f036c48068aed6530d890a0f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:05:26.947916 8074 raft_consensus.cc:2229] T 3d62819dc7d64e45870511ba49657603 P ef968b90aef744228766ae1f30c9878b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:05:26.948400 8074 raft_consensus.cc:2258] T 3d62819dc7d64e45870511ba49657603 P ef968b90aef744228766ae1f30c9878b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:26.948776 8075 raft_consensus.cc:2258] T 3d62819dc7d64e45870511ba49657603 P 7317b99f036c48068aed6530d890a0f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:26.951121 8075 ts_tablet_manager.cc:1905] T 3d62819dc7d64e45870511ba49657603 P 7317b99f036c48068aed6530d890a0f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:05:26.952598 8074 ts_tablet_manager.cc:1905] T 3d62819dc7d64e45870511ba49657603 P ef968b90aef744228766ae1f30c9878b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:05:26.956749 8075 ts_tablet_manager.cc:1918] T 3d62819dc7d64e45870511ba49657603 P 7317b99f036c48068aed6530d890a0f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:05:26.959196 8074 ts_tablet_manager.cc:1918] T 3d62819dc7d64e45870511ba49657603 P ef968b90aef744228766ae1f30c9878b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:05:26.960196 8074 log.cc:1196] T 3d62819dc7d64e45870511ba49657603 P ef968b90aef744228766ae1f30c9878b: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-1/wal/wals/3d62819dc7d64e45870511ba49657603
W20260812 19:05:26.960544 7427 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.130.1:36045 (error 108)
I20260812 19:05:26.961803 8074 ts_tablet_manager.cc:1939] T 3d62819dc7d64e45870511ba49657603 P ef968b90aef744228766ae1f30c9878b: Deleting consensus metadata
I20260812 19:05:26.962028 8075 log.cc:1196] T 3d62819dc7d64e45870511ba49657603 P 7317b99f036c48068aed6530d890a0f9: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561415988404-26120-0/minicluster-data/ts-2/wal/wals/3d62819dc7d64e45870511ba49657603
I20260812 19:05:26.962823 8075 ts_tablet_manager.cc:1939] T 3d62819dc7d64e45870511ba49657603 P 7317b99f036c48068aed6530d890a0f9: Deleting consensus metadata
I20260812 19:05:26.963089 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 7702
I20260812 19:05:26.975267 7423 catalog_manager.cc:4841] TS 7317b99f036c48068aed6530d890a0f9 (127.25.130.3:44489): tablet 3d62819dc7d64e45870511ba49657603 (table test-table [id=04ddfb56381c4442b2dd1fae15d8ce24]) successfully deleted
I20260812 19:05:26.976089 8039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:05:26.975668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:36045 (local address 127.0.0.1:38326)
0812 19:05:26.975904 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0812 19:05:26.975981 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.1:36045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
W20260812 19:05:26.976428 7427 catalog_manager.cc:4584] TS 2b04acc7f41f4c289a25843a0cef10d6 (127.25.130.1:36045): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3d62819dc7d64e45870511ba49657603: Network error: Client connection negotiation failed: client connection to 127.25.130.1:36045: connect: Connection refused (error 111)
W20260812 19:05:26.979439 8039 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:05:26.978730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:39995 (local address 127.0.0.1:43928)
0812 19:05:26.978908 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 19:05:26.978921 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:05:26.979010 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:05:26.979308 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.2:39995: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":95}
I20260812 19:05:26.980219 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 7867
I20260812 19:05:26.991724 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 7405
I20260812 19:05:27.008366 26120 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 19:05:27.353437 26120 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 19:05:27.353648 26120 mini_ranger.cc:66] Stopping Ranger...
I20260812 19:05:27.698872 26120 mini_ranger.cc:68] Stopped Ranger
2026-08-12 19:05:27.699 UTC [6905] LOG: received smart shutdown request
2026-08-12 19:05:27.703 UTC [6905] LOG: background worker "logical replication launcher" (PID 6916) exited with exit code 1
2026-08-12 19:05:27.703 UTC [6910] LOG: shutting down
2026-08-12 19:05:27.865 UTC [6905] LOG: database system is shut down
2026-08-12T19:05:27Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47805 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:05:28 dist-test-slave-49h3 krb5kdc[8086](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:05:28 dist-test-slave-49h3 krb5kdc[8086](info): set up 2 sockets
Aug 12 19:05:28 dist-test-slave-49h3 krb5kdc[8086](info): commencing operation
krb5kdc: starting...
W20260812 19:05:29.039121 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:05:29 dist-test-slave-49h3 krb5kdc[8086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:05:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:05:29Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.25.130.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.25.130.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.25.130.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/rangeradmin_127.25.130.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.25.130.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/rangeradmin_127.25.130.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.25.130.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.25.130.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.25.130.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/rangerlookup_127.25.130.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.25.130.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/rangerlookup_127.25.130.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.25.130.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.25.130.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.25.130.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/HTTP_127.25.130.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.25.130.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/HTTP_127.25.130.20@KRBTEST.COM.keytab.
I20260812 19:05:29.210083 26120 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskt2JiBs/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/postgres -l logfile start
2026-08-12 19:05:31.589 UTC [8122] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 19:05:31.590 UTC [8122] LOG: listening on IPv4 address "127.25.130.20", port 57125
2026-08-12 19:05:31.593 UTC [8122] LOG: listening on Unix socket "/tmp/.s.PGSQL.57125"
2026-08-12 19:05:31.616 UTC [8125] LOG: database system was shut down at 2026-08-12 19:05:30 UTC
2026-08-12 19:05:31.620 UTC [8122] LOG: database system is ready to accept connections
I20260812 19:05:32.593969 26120 mini_postgres.cc:96] Postgres bound to 57125
2026-08-12 19:05:32.602 UTC [8133] FATAL: database "slave" does not exist
127.25.130.20:57125 - accepting connections
I20260812 19:05:32.602903 26120 mini_ranger.cc:161] Starting Ranger...
I20260812 19:05:32.619289 26120 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 19:05:33.076422 26120 mini_ranger.cc:87] Created ranger Postgres database
I20260812 19:05:33.076514 26120 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ranger-admin
2026-08-12 19:05:33,359 [I] DB FLAVOR :POSTGRES
2026-08-12 19:05:33,360 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 19:05:33,360 [I] Checking connection..
2026-08-12 19:05:33,360 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:57125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 19:05:33,619 [I] Checking connection passed.
2026-08-12 19:05:33,619 [I] --------- Verifying version history table ---------
2026-08-12 19:05:33,619 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:57125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 19:05:33,873 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 19:05:33,874 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:57125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 19:05:34,129 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 19:05:34,130 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 19:05:34,130 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:57125/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 19:05:34.379 UTC [8210] WARNING: there is no transaction in progress
2026-08-12 19:05:34,386 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 19:05:34,386 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:57125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 19:05:34,646 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 19:05:34,646 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 19:05:34,646 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:57125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 19:05:34,910 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:57125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 19:05:35,168 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:57125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-49h3.c.gcp-upstream.internal','N') ;"
2026-08-12 19:05:35,396 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:57125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 19:05:35,637 [I] Table x_portal_user does not exist in database ranger
2026-08-12 19:05:35,638 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:57125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 19:05:35,886 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 19:05:35,887 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:57125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 19:05:36,138 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:57125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 19:05:36,417 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 19:05:36,418 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:57125/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 19:05:37.450 UTC [8332] WARNING: there is no transaction in progress
2026-08-12 19:05:37.475 UTC [8332] WARNING: there is no transaction in progress
2026-08-12 19:05:37.492 UTC [8332] WARNING: there is no transaction in progress
2026-08-12 19:05:37.508 UTC [8332] WARNING: there is no transaction in progress
2026-08-12 19:05:37.524 UTC [8332] WARNING: there is no transaction in progress
2026-08-12 19:05:37.541 UTC [8332] WARNING: there is no transaction in progress
2026-08-12 19:05:37.597 UTC [8332] WARNING: there is no transaction in progress
2026-08-12 19:05:37.604 UTC [8332] WARNING: there is no transaction in progress
2026-08-12 19:05:37.618 UTC [8332] WARNING: there is no transaction in progress
2026-08-12 19:05:37.630 UTC [8332] WARNING: there is no transaction in progress
2026-08-12 19:05:37.641 UTC [8332] WARNING: there is no transaction in progress
2026-08-12 19:05:37.655 UTC [8332] WARNING: there is no transaction in progress
2026-08-12 19:05:37.666 UTC [8332] WARNING: there is no transaction in progress
2026-08-12 19:05:37.673 UTC [8332] WARNING: there is no transaction in progress
2026-08-12 19:05:37.679 UTC [8332] WARNING: there is no transaction in progress
2026-08-12 19:05:37.686 UTC [8332] WARNING: there is no transaction in progress
2026-08-12 19:05:38,321 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 19:05:38,322 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:57125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 19:05:38,569 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 19:05:38,570 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:57125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 19:05:38,806 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 19:05:38,806 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:57125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 19:05:39,067 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 19:05:39,067 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:57125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 19:05:39,311 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:57125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 19:05:39,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:57125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-49h3.c.gcp-upstream.internal';"
2026-08-12 19:05:39,803 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 19:05:39,804 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:57125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 19:05:40,040 [I] DB_PATCHES have already been applied
I20260812 19:05:40.046528 26120 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ranger-admin:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskt2JiBs/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 19:05:40.046622 26120 mini_ranger.cc:193] Using host: 127.25.130.20
I20260812 19:05:40.051016 26120 mini_ranger.cc:227] Ranger admin URL: http://127.25.130.20:51387
Aug 12, 2026 7:05:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 7:05:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 7:05:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 7:05:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 7:05:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 7:05:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 7:05:40 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 19:05:41.182816 26120 mini_ranger.cc:160] Time spent starting Ranger: real 8.580s user 0.008s sys 0.004s
Aug 12, 2026 7:06:00 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 7:06:00 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 7:06:00 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 7:06:00 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 7:06:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 7:06:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 7:06:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 7:06:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 7:06:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 7:06:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 7:06:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 7:06:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 7:06:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 7:06:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 7:06:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 7:06:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 7:06:00 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 7:06:01 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 19:06:02.372627 26120 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.25.130.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.25.130.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.25.130.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/rangerkms_127.25.130.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.25.130.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/rangerkms_127.25.130.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.25.130.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.25.130.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.25.130.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/HTTP_127.25.130.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.25.130.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/HTTP_127.25.130.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 19:06:02 dist-test-slave-49h3 krb5kdc[8086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561562, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 19:06:02.458287 26120 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 19:06:02.468485 26120 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 19:06:03.267850 26120 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 19:06:03.267941 26120 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ranger-kms
I20260812 19:06:03.267958 26120 mini_ranger_kms.cc:227] Using postgres at 127.25.130.20:57125
2026-08-12 19:06:03,464 [I] DB FLAVOR :POSTGRES
2026-08-12 19:06:03,465 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 19:06:03,465 [I] Checking connection
2026-08-12 19:06:03,465 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:57125/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 19:06:03,734 [I] connection success
2026-08-12 19:06:03,734 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 19:06:03,734 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 19:06:03,734 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:57125/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 19:06:04,051 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 19:06:04,052 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 19:06:04,052 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 19:06:04,052 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.130.20:57125/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 19:06:04,395 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 19:06:04.895820 26120 mini_ranger_kms.cc:310] Created kms service
I20260812 19:06:05.137341 26120 mini_ranger_kms.cc:326] Created kudu user
I20260812 19:06:05.184554 26120 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 19:06:05 dist-test-slave-49h3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561562, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.25.130.20@KRBTEST.COM
I20260812 19:06:05.590790 26120 mini_ranger_kms.cc:384] Added ranger policy
I20260812 19:06:05.590963 26120 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ranger-kms:/tmp/dist-test-taskt2JiBs/build/debug/bin/postgresql.jar:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskt2JiBs/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 19:06:05.590988 26120 mini_ranger_kms.cc:240] Using host: 127.25.130.20
I20260812 19:06:05.595340 26120 mini_ranger_kms.cc:276] Ranger KMS PID: 8539
I20260812 19:06:05.595435 26120 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.25.130.20:45319
Aug 12, 2026 7:06:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 7:06:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 7:06:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 7:06:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 7:06:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 7:06:07 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260812 19:06:07.841562 26120 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.383s user 0.009s sys 0.173s
I20260812 19:06:07.841686 26120 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 19:06:07.841760 26120 mini_ranger_kms.cc:401] 127.25.130.20:45319/kms/v1/keys
Aug 12 19:06:07 dist-test-slave-49h3 krb5kdc[8086](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786561567, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.130.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 19:06:08 dist-test-slave-49h3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561567, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.130.20@KRBTEST.COM for HTTP/127.25.130.20@KRBTEST.COM
Aug 12 19:06:08 dist-test-slave-49h3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561567, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.130.20@KRBTEST.COM for HTTP/127.25.130.20@KRBTEST.COM
Aug 12 19:06:11 dist-test-slave-49h3 krb5kdc[8086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:06:11.130261 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:36757
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:45981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:36757
--ranger_config_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.130.20:45319/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:06:11.261787 8606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:06:11.262100 8606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:06:11.262140 8606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:06:11.266367 8606 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:06:11.266446 8606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:06:11.266463 8606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:06:11.266479 8606 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:06:11.266494 8606 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:06:11.272238 8606 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:45981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/wal
--ranger_kms_url=127.25.130.20:45319/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:36757
--ranger_config_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:36757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:06:11.273427 8606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:06:11.274602 8606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:11.283751 8612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:11.283777 8613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:11.283977 8606 server_base.cc:1034] running on GCE node
W20260812 19:06:11.283790 8615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:11.284727 8606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:06:11.285965 8606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:06:11.287166 8606 hybrid_clock.cc:648] HybridClock initialized: now 1786561571287146 us; error 35 us; skew 500 ppm
Aug 12 19:06:11 dist-test-slave-49h3 krb5kdc[8086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561571, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:11.290091 8606 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:06:11.291213 8606 webserver.cc:460] Webserver started at http://127.25.130.62:35201/ using document root <none> and password file <none>
I20260812 19:06:11.291887 8606 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:11.291961 8606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:11.292203 8606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 19:06:11 dist-test-slave-49h3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561571, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for HTTP/127.25.130.20@KRBTEST.COM
I20260812 19:06:11.342177 8606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "cd5f9ca68867402196c0be762560c4ec"
format_stamp: "Formatted at 2026-08-12 19:06:11 on dist-test-slave-49h3"
server_key: "74724a5707e80e0bb7c4c02917d1b5ed"
server_key_iv: "4afea98c4789870ac7576db5a7b09465"
server_key_version: "kuduclusterkey@0"
I20260812 19:06:11.342942 8606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "cd5f9ca68867402196c0be762560c4ec"
format_stamp: "Formatted at 2026-08-12 19:06:11 on dist-test-slave-49h3"
server_key: "74724a5707e80e0bb7c4c02917d1b5ed"
server_key_iv: "4afea98c4789870ac7576db5a7b09465"
server_key_version: "kuduclusterkey@0"
I20260812 19:06:11.348292 8606 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 19:06:11.366962 8622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:11.368633 8606 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 19:06:11.368799 8606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "cd5f9ca68867402196c0be762560c4ec"
format_stamp: "Formatted at 2026-08-12 19:06:11 on dist-test-slave-49h3"
server_key: "74724a5707e80e0bb7c4c02917d1b5ed"
server_key_iv: "4afea98c4789870ac7576db5a7b09465"
server_key_version: "kuduclusterkey@0"
I20260812 19:06:11.368978 8606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:11.395082 8606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:11.398464 8606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:06:11.398684 8606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:11.407581 8606 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:36757
I20260812 19:06:11.407586 8678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:36757 every 8 connection(s)
I20260812 19:06:11.409044 8606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:06:11.409749 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 8606
I20260812 19:06:11.409878 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:06:11.410207 26120 external_mini_cluster.cc:1428] Setting key 5e58607d2dc224219deeea033dfb9fc7
I20260812 19:06:11.412684 8679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:11.419384 8679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec: Bootstrap starting.
Aug 12 19:06:11 dist-test-slave-49h3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:06:11.422168 8679 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:11.423183 8679 log.cc:824] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec: Log is configured to *not* fsync() on all Append() calls
I20260812 19:06:11.425827 8679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec: No bootstrap required, opened a new log
I20260812 19:06:11.428560 8683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:11.411724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32800 (local address 127.25.130.62:36757)
0812 19:06:11.412249 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:11.412269 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:11.412318 (+ 49us) server_negotiation.cc:407] Connection header received
0812 19:06:11.413207 (+ 889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:11.413234 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:11.413743 (+ 509us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:11.414143 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:11.415060 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:11.416663 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:11.417932 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:11.418681 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:11.418848 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:11.422065 (+ 3217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:11.422073 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:11.422092 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:11.422145 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:11.424863 (+ 2718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:11.425475 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:11.425481 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:11.425488 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:11.425634 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:11.425961 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:11.425965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:11.425967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:11.426541 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:11.426765 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:11.427089 (+ 324us) server_negotiation.cc:299] Negotiation successful
0812 19:06:11.427579 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":157,"threads_started":1}
I20260812 19:06:11.429198 8679 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd5f9ca68867402196c0be762560c4ec" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 36757 } }
I20260812 19:06:11.429461 8679 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:11.429538 8679 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd5f9ca68867402196c0be762560c4ec, State: Initialized, Role: FOLLOWER
I20260812 19:06:11.430078 8679 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd5f9ca68867402196c0be762560c4ec" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 36757 } }
I20260812 19:06:11.430249 8679 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:06:11.430343 8679 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:06:11.430444 8679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:11.431566 8679 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd5f9ca68867402196c0be762560c4ec" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 36757 } }
I20260812 19:06:11.431900 8679 leader_election.cc:302] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd5f9ca68867402196c0be762560c4ec; no voters:
I20260812 19:06:11.432205 8679 leader_election.cc:288] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:06:11.432361 8685 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:11.432531 8685 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [term 1 LEADER]: Becoming Leader. State: Replica: cd5f9ca68867402196c0be762560c4ec, State: Running, Role: LEADER
I20260812 19:06:11.432890 8685 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd5f9ca68867402196c0be762560c4ec" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 36757 } }
I20260812 19:06:11.433557 8679 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:06:11.434855 8686 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd5f9ca68867402196c0be762560c4ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd5f9ca68867402196c0be762560c4ec" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 36757 } } }
I20260812 19:06:11.434986 8686 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [sys.catalog]: This master's current role is: LEADER
I20260812 19:06:11.435406 8687 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd5f9ca68867402196c0be762560c4ec. Latest consensus state: current_term: 1 leader_uuid: "cd5f9ca68867402196c0be762560c4ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd5f9ca68867402196c0be762560c4ec" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 36757 } } }
I20260812 19:06:11.435552 8687 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [sys.catalog]: This master's current role is: LEADER
I20260812 19:06:11.435868 8694 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:06:11.435918 8679 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 19:06:11.442696 8694 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:06:11.451304 8694 catalog_manager.cc:1347] Generated new cluster ID: d399a36893ab43b18f1ac1d413649c84
I20260812 19:06:11.451473 8694 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:06:11.476478 8694 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:06:11.478116 8694 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:06:11.489507 8694 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec: Generated new TSK 0
I20260812 19:06:11.490610 8694 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
I20260812 19:06:12.238806 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.130.20:45319/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:0
--local_ip_for_outbound_sockets=127.25.130.1
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:36757
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:45981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:06:12.359140 8736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:06:12.359427 8736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:06:12.359491 8736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:06:12.363888 8736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:06:12.363966 8736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:06:12.364049 8736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:06:12.370466 8736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:45981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.25.130.20:45319/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:36757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:06:12.371650 8736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:06:12.372746 8736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:12.383448 8746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:12.383412 8742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:12.383397 8741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:12.383934 8736 server_base.cc:1034] running on GCE node
I20260812 19:06:12.384375 8736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:06:12.385079 8736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:06:12.386250 8736 hybrid_clock.cc:648] HybridClock initialized: now 1786561572386227 us; error 40 us; skew 500 ppm
Aug 12 19:06:12 dist-test-slave-49h3 krb5kdc[8086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561572, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:12.389357 8736 init.cc:377] Logged in from keytab as kudu/127.25.130.1@KRBTEST.COM (short username kudu)
I20260812 19:06:12.390702 8736 webserver.cc:460] Webserver started at http://127.25.130.1:44147/ using document root <none> and password file <none>
I20260812 19:06:12.391597 8736 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:12.391716 8736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:12.392002 8736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 19:06:12 dist-test-slave-49h3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561572, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for HTTP/127.25.130.20@KRBTEST.COM
I20260812 19:06:12.408085 8736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/data/instance:
uuid: "0d712c5ebf36408bae8af256d49eb843"
format_stamp: "Formatted at 2026-08-12 19:06:12 on dist-test-slave-49h3"
server_key: "ad5f8a59f262352076b99bcdee2a53b1"
server_key_iv: "8edcd218c41dc594af96334b98fe3903"
server_key_version: "kuduclusterkey@0"
I20260812 19:06:12.408994 8736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance:
uuid: "0d712c5ebf36408bae8af256d49eb843"
format_stamp: "Formatted at 2026-08-12 19:06:12 on dist-test-slave-49h3"
server_key: "ad5f8a59f262352076b99bcdee2a53b1"
server_key_iv: "8edcd218c41dc594af96334b98fe3903"
server_key_version: "kuduclusterkey@0"
I20260812 19:06:12.424957 8736 fs_manager.cc:691] Time spent creating directory manager: real 0.016s user 0.004s sys 0.000s
I20260812 19:06:12.447408 8758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:12.449304 8736 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260812 19:06:12.449589 8736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "0d712c5ebf36408bae8af256d49eb843"
format_stamp: "Formatted at 2026-08-12 19:06:12 on dist-test-slave-49h3"
server_key: "ad5f8a59f262352076b99bcdee2a53b1"
server_key_iv: "8edcd218c41dc594af96334b98fe3903"
server_key_version: "kuduclusterkey@0"
I20260812 19:06:12.449847 8736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:12.466114 8736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:12.469892 8736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:06:12.470167 8736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:12.471031 8736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:06:12.472177 8736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:06:12.472287 8736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:12.472409 8736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:06:12.472496 8736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:12.484565 8736 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:36833
I20260812 19:06:12.485939 8736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:06:12.488351 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 8736
I20260812 19:06:12.488484 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance
I20260812 19:06:12.488778 26120 external_mini_cluster.cc:1428] Setting key 8775a073d8481f0a5c93b1e7c400799b
I20260812 19:06:12.498286 8887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:36833 every 8 connection(s)
WARNING: no policy specified for kudu/127.25.130.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.2@KRBTEST.COM" created.
Aug 12 19:06:12 dist-test-slave-49h3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561572, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:06:12.553583 8892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:12.502588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:44781 (local address 127.25.130.62:36757)
0812 19:06:12.511316 (+ 8728us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:12.511322 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:12.515755 (+ 4433us) server_negotiation.cc:407] Connection header received
0812 19:06:12.515821 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:12.515824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:12.515895 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:12.515993 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:12.518809 (+ 2816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:12.519686 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:12.538462 (+ 18776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:12.538855 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:12.538965 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:12.547715 (+ 8750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:12.547720 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:12.547723 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:12.547760 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:12.549857 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:12.550942 (+ 1085us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:12.550947 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:12.550950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:12.551059 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:12.551616 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:12.551620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:12.551622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:12.551951 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:12.552039 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:12.553090 (+ 1051us) server_negotiation.cc:299] Negotiation successful
0812 19:06:12.553249 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8606,"thread_start_us":112,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
I20260812 19:06:12.557180 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.130.20:45319/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:0
--local_ip_for_outbound_sockets=127.25.130.2
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:36757
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:45981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:06:12.561347 8893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:12.503361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:36757 (local address 127.25.130.1:44781)
0812 19:06:12.513924 (+ 10563us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:12.514025 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:12.515343 (+ 1318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:12.517341 (+ 1998us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:12.517353 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:12.517779 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:06:12.518575 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:12.518599 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:12.519878 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:12.519882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:12.538184 (+ 18302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:12.538214 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:12.539289 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:12.539295 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:12.539643 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:12.542315 (+ 2672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:06:12.542336 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:06:12.547175 (+ 4839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:06:12.550177 (+ 3002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:12.550186 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:12.550207 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:12.550677 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:12.551183 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:12.551186 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:12.551188 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:12.551498 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:12.552282 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:06:12.552290 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:06:12.552707 (+ 417us) client_negotiation.cc:769] Sending connection context
0812 19:06:12.554210 (+ 1503us) client_negotiation.cc:240] Negotiation successful
0812 19:06:12.554644 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10209,"thread_start_us":168,"threads_started":1}
I20260812 19:06:12.563464 8888 heartbeater.cc:347] Connected to a master server at 127.25.130.62:36757
I20260812 19:06:12.563783 8888 heartbeater.cc:464] Registering TS with master...
I20260812 19:06:12.564467 8888 heartbeater.cc:511] Master 127.25.130.62:36757 requested a full tablet report, sending...
I20260812 19:06:12.566642 8641 ts_manager.cc:194] Registered new tserver with Master: 0d712c5ebf36408bae8af256d49eb843 (127.25.130.1:36833)
I20260812 19:06:12.568306 8641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.1@KRBTEST.COM'} at 127.25.130.1:44781
Aug 12 19:06:12 dist-test-slave-49h3 krb5kdc[8086](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786561572, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 19:06:12.741523 8898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:06:12.742018 8898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:06:12.742147 8898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:06:12.748347 8898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:06:12.748507 8898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:06:12.748646 8898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:06:12.756839 8898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:45981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.25.130.20:45319/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:36757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:06:12.758456 8898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:06:12.759948 8898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:12.774799 8909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:12.775264 8905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:12.775785 8904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:12.775918 8898 server_base.cc:1034] running on GCE node
I20260812 19:06:12.776608 8898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:06:12.777555 8898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:06:12.780855 8898 hybrid_clock.cc:648] HybridClock initialized: now 1786561572780751 us; error 105 us; skew 500 ppm
Aug 12 19:06:12 dist-test-slave-49h3 krb5kdc[8086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561572, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:12.785008 8898 init.cc:377] Logged in from keytab as kudu/127.25.130.2@KRBTEST.COM (short username kudu)
I20260812 19:06:12.786540 8898 webserver.cc:460] Webserver started at http://127.25.130.2:40393/ using document root <none> and password file <none>
I20260812 19:06:12.787467 8898 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:12.787659 8898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:12.788000 8898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 19:06:12 dist-test-slave-49h3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561572, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for HTTP/127.25.130.20@KRBTEST.COM
I20260812 19:06:12.810968 8898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/data/instance:
uuid: "e249e52fffd0421b91835b970bdbe5f0"
format_stamp: "Formatted at 2026-08-12 19:06:12 on dist-test-slave-49h3"
server_key: "1924fe57ce16f0d86445011cc2890d0b"
server_key_iv: "ec9051748b3a830549f5352df81aa5ff"
server_key_version: "kuduclusterkey@0"
I20260812 19:06:12.815584 8898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance:
uuid: "e249e52fffd0421b91835b970bdbe5f0"
format_stamp: "Formatted at 2026-08-12 19:06:12 on dist-test-slave-49h3"
server_key: "1924fe57ce16f0d86445011cc2890d0b"
server_key_iv: "ec9051748b3a830549f5352df81aa5ff"
server_key_version: "kuduclusterkey@0"
I20260812 19:06:12.823406 8898 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.000s
I20260812 19:06:12.848626 8919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:12.850369 8898 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:06:12.850523 8898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "e249e52fffd0421b91835b970bdbe5f0"
format_stamp: "Formatted at 2026-08-12 19:06:12 on dist-test-slave-49h3"
server_key: "1924fe57ce16f0d86445011cc2890d0b"
server_key_iv: "ec9051748b3a830549f5352df81aa5ff"
server_key_version: "kuduclusterkey@0"
I20260812 19:06:12.850672 8898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:12.876571 8898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:12.880669 8898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:06:12.880944 8898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:12.881914 8898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:06:12.883288 8898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:06:12.883394 8898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:12.883489 8898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:06:12.883560 8898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:12.897243 8898 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:33931
I20260812 19:06:12.898574 8898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:06:12.899453 9042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:33931 every 8 connection(s)
I20260812 19:06:12.904772 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 8898
I20260812 19:06:12.904922 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance
I20260812 19:06:12.905256 26120 external_mini_cluster.cc:1428] Setting key 330ed47de43cdaf24e6f2b36e8a32721
Aug 12 19:06:12 dist-test-slave-49h3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561572, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:06:12.923802 8892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:12.903568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:34727 (local address 127.25.130.62:36757)
0812 19:06:12.903701 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:12.903705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:12.905122 (+ 1417us) server_negotiation.cc:407] Connection header received
0812 19:06:12.907750 (+ 2628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:12.907755 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:12.907810 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:12.907858 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:12.910920 (+ 3062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:12.911724 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:12.913441 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:12.913828 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:12.913906 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:12.917075 (+ 3169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:12.917079 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:12.917082 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:12.917112 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:12.918815 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:12.921785 (+ 2970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:12.921790 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:12.921792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:12.921911 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:12.922376 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:12.922380 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:12.922382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:12.922639 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:12.922724 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:12.923461 (+ 737us) server_negotiation.cc:299] Negotiation successful
0812 19:06:12.923631 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 19:06:12.926067 9046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:12.902134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:36757 (local address 127.25.130.2:34727)
0812 19:06:12.904802 (+ 2668us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:12.904935 (+ 133us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:12.907473 (+ 2538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:12.909624 (+ 2151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:12.909635 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:12.910017 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:06:12.910733 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:12.910753 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:12.911898 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:12.911901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:12.913288 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:12.913300 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:12.914134 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:12.914138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:12.914348 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:12.915071 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:06:12.915092 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:06:12.916850 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:06:12.918978 (+ 2128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:12.918984 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:12.919004 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:12.919366 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:12.922036 (+ 2670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:12.922039 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:12.922041 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:12.922265 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:12.922801 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:06:12.922809 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:06:12.923170 (+ 361us) client_negotiation.cc:769] Sending connection context
0812 19:06:12.924292 (+ 1122us) client_negotiation.cc:240] Negotiation successful
0812 19:06:12.924704 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2437,"thread_start_us":155,"threads_started":1}
WARNING: no policy specified for kudu/127.25.130.3@KRBTEST.COM; defaulting to no policy
I20260812 19:06:12.933262 9044 heartbeater.cc:347] Connected to a master server at 127.25.130.62:36757
I20260812 19:06:12.933619 9044 heartbeater.cc:464] Registering TS with master...
I20260812 19:06:12.934283 9044 heartbeater.cc:511] Master 127.25.130.62:36757 requested a full tablet report, sending...
I20260812 19:06:12.935809 8641 ts_manager.cc:194] Registered new tserver with Master: e249e52fffd0421b91835b970bdbe5f0 (127.25.130.2:33931)
I20260812 19:06:12.936599 8641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.2@KRBTEST.COM'} at 127.25.130.2:34727
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
I20260812 19:06:12.968997 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.130.20:45319/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:0
--local_ip_for_outbound_sockets=127.25.130.3
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:36757
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:45981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 19:06:13 dist-test-slave-49h3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561572, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for HTTP/127.25.130.20@KRBTEST.COM
W20260812 19:06:13.160708 9051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:06:13.161229 9051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:06:13.161338 9051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:06:13.167673 9051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:06:13.167860 9051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:06:13.167997 9051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:06:13.176633 9051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:45981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.25.130.20:45319/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:36757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:06:13.178382 9051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:06:13.179831 9051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:13.194037 9057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:13.194063 9059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:13.195240 9056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:13.195749 9051 server_base.cc:1034] running on GCE node
I20260812 19:06:13.196398 9051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:06:13.197293 9051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:06:13.212803 9051 hybrid_clock.cc:648] HybridClock initialized: now 1786561573212678 us; error 130 us; skew 500 ppm
Aug 12 19:06:13 dist-test-slave-49h3 krb5kdc[8086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561573, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:13.217291 9051 init.cc:377] Logged in from keytab as kudu/127.25.130.3@KRBTEST.COM (short username kudu)
I20260812 19:06:13.218660 9051 webserver.cc:460] Webserver started at http://127.25.130.3:37813/ using document root <none> and password file <none>
I20260812 19:06:13.219512 9051 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:13.219686 9051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:13.220005 9051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 19:06:13 dist-test-slave-49h3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561573, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for HTTP/127.25.130.20@KRBTEST.COM
I20260812 19:06:13.243017 9051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/data/instance:
uuid: "0da1050c29b64f3d87984891059b70c6"
format_stamp: "Formatted at 2026-08-12 19:06:13 on dist-test-slave-49h3"
server_key: "66b5693dc02d5e545a4f937aaea1c419"
server_key_iv: "21f48aafe2005a7db01958228b85e204"
server_key_version: "kuduclusterkey@0"
I20260812 19:06:13.243935 9051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance:
uuid: "0da1050c29b64f3d87984891059b70c6"
format_stamp: "Formatted at 2026-08-12 19:06:13 on dist-test-slave-49h3"
server_key: "66b5693dc02d5e545a4f937aaea1c419"
server_key_iv: "21f48aafe2005a7db01958228b85e204"
server_key_version: "kuduclusterkey@0"
I20260812 19:06:13.250006 9051 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260812 19:06:13.280042 9069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:13.281910 9051 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.003s sys 0.000s
I20260812 19:06:13.282075 9051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "0da1050c29b64f3d87984891059b70c6"
format_stamp: "Formatted at 2026-08-12 19:06:13 on dist-test-slave-49h3"
server_key: "66b5693dc02d5e545a4f937aaea1c419"
server_key_iv: "21f48aafe2005a7db01958228b85e204"
server_key_version: "kuduclusterkey@0"
I20260812 19:06:13.282222 9051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:13.320542 9051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:13.324721 9051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:06:13.324934 9051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:13.325739 9051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:06:13.326908 9051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:06:13.326980 9051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:13.327117 9051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:06:13.327167 9051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:13.346192 9051 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:37661
I20260812 19:06:13.346380 9197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:37661 every 8 connection(s)
I20260812 19:06:13.347586 9051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:06:13.348052 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 9051
I20260812 19:06:13.348177 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance
I20260812 19:06:13.348476 26120 external_mini_cluster.cc:1428] Setting key 4c9f4317ea07747e7065b950848bee33
Aug 12 19:06:13 dist-test-slave-49h3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561573, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:06:13.366259 8892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:13.351499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:41441 (local address 127.25.130.62:36757)
0812 19:06:13.351642 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:13.351647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:13.352740 (+ 1093us) server_negotiation.cc:407] Connection header received
0812 19:06:13.354001 (+ 1261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:13.354005 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:13.354061 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:13.354158 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:13.355757 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:13.356613 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:13.358551 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:13.358887 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:13.358997 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:13.362592 (+ 3595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:13.362596 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:13.362598 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:13.362629 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:13.364102 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:13.364617 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:13.364621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:13.364623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:13.364701 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:13.365043 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:13.365046 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:13.365048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:13.365243 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:13.365329 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:13.365942 (+ 613us) server_negotiation.cc:299] Negotiation successful
0812 19:06:13.366092 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 19:06:13.367218 9201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:13.351954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:36757 (local address 127.25.130.3:41441)
0812 19:06:13.352577 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:13.352638 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:13.353765 (+ 1127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:13.354348 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:13.354358 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:13.354777 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:06:13.355548 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:13.355571 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:13.356774 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:13.356778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:13.358388 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:13.358402 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:13.359078 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:13.359083 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:13.359313 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:13.360069 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:06:13.360091 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:06:13.362375 (+ 2284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:06:13.364219 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:13.364225 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:13.364243 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:13.364507 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:13.364782 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:13.364785 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:13.364787 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:13.364944 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:13.365346 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:06:13.365353 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:06:13.365678 (+ 325us) client_negotiation.cc:769] Sending connection context
0812 19:06:13.365857 (+ 179us) client_negotiation.cc:240] Negotiation successful
0812 19:06:13.366226 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":136,"threads_started":1}
I20260812 19:06:13.368528 9199 heartbeater.cc:347] Connected to a master server at 127.25.130.62:36757
I20260812 19:06:13.368793 9199 heartbeater.cc:464] Registering TS with master...
I20260812 19:06:13.369335 9199 heartbeater.cc:511] Master 127.25.130.62:36757 requested a full tablet report, sending...
I20260812 19:06:13.370466 8641 ts_manager.cc:194] Registered new tserver with Master: 0da1050c29b64f3d87984891059b70c6 (127.25.130.3:37661)
I20260812 19:06:13.371047 8641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.3@KRBTEST.COM'} at 127.25.130.3:41441
I20260812 19:06:13.374891 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:06:13.375669 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 8736
I20260812 19:06:13.404171 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 8898
I20260812 19:06:13.418453 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 9051
I20260812 19:06:13.426710 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 8606
I20260812 19:06:13.436184 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:36757
--webserver_interface=127.25.130.62
--webserver_port=35201
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:45981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:36757
--ranger_config_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.130.20:45319/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.25.130.20:45319/kms with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:06:13.550802 9202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:06:13.551088 9202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:06:13.551131 9202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:06:13.555331 9202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:06:13.555404 9202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:06:13.555423 9202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:06:13.555437 9202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:06:13.555451 9202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:06:13.561084 9202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:45981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.25.130.20:45319/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:36757
--ranger_config_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:36757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=35201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:06:13.562173 9202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:06:13.563244 9202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:13.571736 9207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:13.571794 9208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:13.571736 9210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:13.572192 9202 server_base.cc:1034] running on GCE node
I20260812 19:06:13.572806 9202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:06:13.573983 9202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:06:13.575146 9202 hybrid_clock.cc:648] HybridClock initialized: now 1786561573575129 us; error 34 us; skew 500 ppm
Aug 12 19:06:13 dist-test-slave-49h3 krb5kdc[8086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561573, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:13.577877 9202 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:06:13.578752 9202 webserver.cc:460] Webserver started at http://127.25.130.62:35201/ using document root <none> and password file <none>
I20260812 19:06:13.579252 9202 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:13.579392 9202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 19:06:13 dist-test-slave-49h3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561573, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for HTTP/127.25.130.20@KRBTEST.COM
I20260812 19:06:13.621668 9202 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:06:13.623947 9222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:13.625253 9202 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:06:13.625387 9202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "cd5f9ca68867402196c0be762560c4ec"
format_stamp: "Formatted at 2026-08-12 19:06:11 on dist-test-slave-49h3"
server_key: "74724a5707e80e0bb7c4c02917d1b5ed"
server_key_iv: "4afea98c4789870ac7576db5a7b09465"
server_key_version: "kuduclusterkey@0"
I20260812 19:06:13.625968 9202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:13.656752 9202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:13.659490 9202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:06:13.659682 9202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:13.666853 9202 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:36757
I20260812 19:06:13.666872 9281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:36757 every 8 connection(s)
I20260812 19:06:13.667922 9202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:06:13.672142 9282 sys_catalog.cc:263] Verifying existing consensus state
I20260812 19:06:13.673905 9282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec: Bootstrap starting.
I20260812 19:06:13.674551 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 9202
I20260812 19:06:13.674968 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.130.20:45319/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:36833
--local_ip_for_outbound_sockets=127.25.130.1
--tserver_master_addrs=127.25.130.62:36757
--webserver_port=44147
--webserver_interface=127.25.130.1
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:45981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.25.130.20:45319/kms with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:06:13.690582 9282 log.cc:824] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec: Log is configured to *not* fsync() on all Append() calls
I20260812 19:06:13.697330 9282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:06:13.697772 9282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec: Bootstrap complete.
I20260812 19:06:13.701736 9282 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd5f9ca68867402196c0be762560c4ec" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 36757 } }
I20260812 19:06:13.702132 9282 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd5f9ca68867402196c0be762560c4ec, State: Initialized, Role: FOLLOWER
I20260812 19:06:13.702657 9282 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd5f9ca68867402196c0be762560c4ec" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 36757 } }
I20260812 19:06:13.702786 9282 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:06:13.702852 9282 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:06:13.702955 9282 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:13.704870 9282 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd5f9ca68867402196c0be762560c4ec" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 36757 } }
I20260812 19:06:13.705184 9282 leader_election.cc:302] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd5f9ca68867402196c0be762560c4ec; no voters:
I20260812 19:06:13.705535 9282 leader_election.cc:288] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 19:06:13.705631 9286 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:06:13.705823 9286 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [term 2 LEADER]: Becoming Leader. State: Replica: cd5f9ca68867402196c0be762560c4ec, State: Running, Role: LEADER
I20260812 19:06:13.706223 9286 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd5f9ca68867402196c0be762560c4ec" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 36757 } }
I20260812 19:06:13.706840 9282 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:06:13.707579 9287 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "cd5f9ca68867402196c0be762560c4ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd5f9ca68867402196c0be762560c4ec" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 36757 } } }
I20260812 19:06:13.707712 9287 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [sys.catalog]: This master's current role is: LEADER
I20260812 19:06:13.708320 9288 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd5f9ca68867402196c0be762560c4ec. Latest consensus state: current_term: 2 leader_uuid: "cd5f9ca68867402196c0be762560c4ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd5f9ca68867402196c0be762560c4ec" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 36757 } } }
I20260812 19:06:13.708416 9288 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec [sys.catalog]: This master's current role is: LEADER
I20260812 19:06:13.708740 9296 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:06:13.709138 9282 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 19:06:13.715006 9296 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:06:13.715813 9296 catalog_manager.cc:1259] Loaded cluster ID: d399a36893ab43b18f1ac1d413649c84
I20260812 19:06:13.715917 9296 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:06:13.718712 9296 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:06:13.719591 9296 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec: Loaded TSK: 0
I20260812 19:06:13.720989 9296 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 19:06:13.820277 9284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:06:13.820762 9284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:06:13.820883 9284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:06:13.827103 9284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:06:13.827204 9284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:06:13.827291 9284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:06:13.834913 9284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:45981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.25.130.20:45319/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:36833
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=44147
--tserver_master_addrs=127.25.130.62:36757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:06:13.836413 9284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:06:13.837770 9284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:06:13.851023 9284 server_base.cc:1034] running on GCE node
W20260812 19:06:13.850976 9326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:13.850950 9324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:13.851339 9323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:13.851969 9284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:06:13.852799 9284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:06:13.853991 9284 hybrid_clock.cc:648] HybridClock initialized: now 1786561573853961 us; error 47 us; skew 500 ppm
Aug 12 19:06:13 dist-test-slave-49h3 krb5kdc[8086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561573, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:13.857733 9284 init.cc:377] Logged in from keytab as kudu/127.25.130.1@KRBTEST.COM (short username kudu)
I20260812 19:06:13.858984 9284 webserver.cc:460] Webserver started at http://127.25.130.1:44147/ using document root <none> and password file <none>
I20260812 19:06:13.859745 9284 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:13.859864 9284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 19:06:13 dist-test-slave-49h3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561573, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for HTTP/127.25.130.20@KRBTEST.COM
I20260812 19:06:13.893596 9284 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:06:13.903705 9336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:13.905570 9284 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.003s sys 0.000s
I20260812 19:06:13.905736 9284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "0d712c5ebf36408bae8af256d49eb843"
format_stamp: "Formatted at 2026-08-12 19:06:12 on dist-test-slave-49h3"
server_key: "ad5f8a59f262352076b99bcdee2a53b1"
server_key_iv: "8edcd218c41dc594af96334b98fe3903"
server_key_version: "kuduclusterkey@0"
I20260812 19:06:13.906487 9284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:13.919765 9284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:13.923574 9284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:06:13.923774 9284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:13.924540 9284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:06:13.925786 9284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:06:13.925905 9284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:13.926064 9284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:06:13.926157 9284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:13.941440 9284 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:36833
I20260812 19:06:13.942988 9284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:06:13.945235 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 9284
I20260812 19:06:13.945816 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.130.20:45319/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:33931
--local_ip_for_outbound_sockets=127.25.130.2
--tserver_master_addrs=127.25.130.62:36757
--webserver_port=40393
--webserver_interface=127.25.130.2
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:45981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.25.130.20:45319/kms with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:06:13.948998 9469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:36833 every 8 connection(s)
Aug 12 19:06:13 dist-test-slave-49h3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561573, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:06:13.974392 9474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:13.949482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:36757 (local address 127.25.130.1:50807)
0812 19:06:13.955961 (+ 6479us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:13.956048 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:13.957259 (+ 1211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:13.959094 (+ 1835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:13.959103 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:13.959461 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:06:13.960152 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:13.960174 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:13.962139 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:13.962142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:13.963544 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:13.963556 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:13.964854 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:13.964859 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:13.965053 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:13.965797 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:06:13.965820 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:06:13.967515 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:06:13.970462 (+ 2947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:13.970469 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:13.970487 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:13.970911 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:13.971423 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:13.971427 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:13.971429 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:13.971627 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:13.972484 (+ 857us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:06:13.972491 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:06:13.972805 (+ 314us) client_negotiation.cc:769] Sending connection context
0812 19:06:13.973005 (+ 200us) client_negotiation.cc:240] Negotiation successful
0812 19:06:13.973349 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6154,"thread_start_us":162,"threads_started":1}
I20260812 19:06:13.976004 9475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:13.951045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:50807 (local address 127.25.130.62:36757)
0812 19:06:13.956537 (+ 5492us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:13.956563 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:13.956623 (+ 60us) server_negotiation.cc:407] Connection header received
0812 19:06:13.957991 (+ 1368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:13.958006 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:13.958445 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:13.958807 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:13.960356 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:13.961973 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:13.963712 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:13.964457 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:13.964614 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:13.967791 (+ 3177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:13.967799 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:13.967818 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:13.967855 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:13.970284 (+ 2429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:13.971107 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:13.971114 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:13.971120 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:13.971295 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:13.971764 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:13.971768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:13.971769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:13.972289 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:13.972423 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:13.974797 (+ 2374us) server_negotiation.cc:299] Negotiation successful
0812 19:06:13.975114 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5279,"thread_start_us":122,"threads_started":1}
I20260812 19:06:13.979255 9471 heartbeater.cc:347] Connected to a master server at 127.25.130.62:36757
I20260812 19:06:13.979534 9471 heartbeater.cc:464] Registering TS with master...
I20260812 19:06:13.980141 9471 heartbeater.cc:511] Master 127.25.130.62:36757 requested a full tablet report, sending...
W20260812 19:06:13.982625 9471 heartbeater.cc:644] Failed to heartbeat to 127.25.130.62:36757 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 19:06:13.982769 9471 heartbeater.cc:464] Registering TS with master...
I20260812 19:06:13.982950 9471 heartbeater.cc:511] Master 127.25.130.62:36757 requested a full tablet report, sending...
I20260812 19:06:13.984066 9471 heartbeater.cc:464] Registering TS with master...
I20260812 19:06:13.984249 9471 heartbeater.cc:511] Master 127.25.130.62:36757 requested a full tablet report, sending...
W20260812 19:06:13.985113 9471 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 19:06:14.090955 9473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:06:14.091292 9473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:06:14.091434 9473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:06:14.096263 9473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:06:14.096352 9473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:06:14.096436 9473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:06:14.104311 9473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:45981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.25.130.20:45319/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:33931
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=40393
--tserver_master_addrs=127.25.130.62:36757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:06:14.106005 9473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:06:14.107506 9473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:14.121150 9489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:14.121150 9484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:14.121150 9487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:14.121738 9473 server_base.cc:1034] running on GCE node
I20260812 19:06:14.122153 9473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:06:14.122843 9473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:06:14.124003 9473 hybrid_clock.cc:648] HybridClock initialized: now 1786561574123968 us; error 54 us; skew 500 ppm
Aug 12 19:06:14 dist-test-slave-49h3 krb5kdc[8086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561574, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:14.127249 9473 init.cc:377] Logged in from keytab as kudu/127.25.130.2@KRBTEST.COM (short username kudu)
I20260812 19:06:14.128492 9473 webserver.cc:460] Webserver started at http://127.25.130.2:40393/ using document root <none> and password file <none>
I20260812 19:06:14.129346 9473 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:14.129566 9473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 19:06:14 dist-test-slave-49h3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561574, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for HTTP/127.25.130.20@KRBTEST.COM
I20260812 19:06:14.151775 9473 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.003s
I20260812 19:06:14.163518 9497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:14.165436 9473 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.003s sys 0.000s
I20260812 19:06:14.165685 9473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "e249e52fffd0421b91835b970bdbe5f0"
format_stamp: "Formatted at 2026-08-12 19:06:12 on dist-test-slave-49h3"
server_key: "1924fe57ce16f0d86445011cc2890d0b"
server_key_iv: "ec9051748b3a830549f5352df81aa5ff"
server_key_version: "kuduclusterkey@0"
I20260812 19:06:14.166510 9473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:14.179800 9473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:14.183156 9473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:06:14.183408 9473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:14.184187 9473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:06:14.185317 9473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:06:14.185387 9473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:14.185477 9473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:06:14.185518 9473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260812 19:06:14.199155 9473 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:33931
I20260812 19:06:14.200361 9473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:06:14.205178 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 9473
I20260812 19:06:14.205729 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.130.20:45319/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:37661
--local_ip_for_outbound_sockets=127.25.130.3
--tserver_master_addrs=127.25.130.62:36757
--webserver_port=37813
--webserver_interface=127.25.130.3
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:45981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.25.130.20:45319/kms with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:06:14.206339 9637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:33931 every 8 connection(s)
Aug 12 19:06:14 dist-test-slave-49h3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561574, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:06:14.235872 9475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:14.210182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:35745 (local address 127.25.130.62:36757)
0812 19:06:14.210308 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:14.210311 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:14.218246 (+ 7935us) server_negotiation.cc:407] Connection header received
0812 19:06:14.219431 (+ 1185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:14.219435 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:14.219507 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:14.219584 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:14.221922 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:14.222779 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:14.224561 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:14.224938 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:14.225030 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:14.231762 (+ 6732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:14.231766 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:14.231769 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:14.231803 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:14.233527 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:14.234130 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:14.234134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:14.234136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:14.234238 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:14.234669 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:14.234672 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:14.234674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:14.234881 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:14.234969 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:14.235531 (+ 562us) server_negotiation.cc:299] Negotiation successful
0812 19:06:14.235694 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 19:06:14.236914 9642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:14.208874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:36757 (local address 127.25.130.2:35745)
0812 19:06:14.218020 (+ 9146us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:14.218094 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:14.219174 (+ 1080us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:14.220483 (+ 1309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:14.220495 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:14.220915 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:06:14.221710 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:14.221732 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:14.222962 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:14.222966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:14.224400 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:14.224412 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:14.225275 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:14.225278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:14.225530 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:14.228657 (+ 3127us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:06:14.228683 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:06:14.231535 (+ 2852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:06:14.233664 (+ 2129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:14.233669 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:14.233685 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:14.233984 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:14.234358 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:14.234361 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:14.234364 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:14.234550 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:14.234988 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:06:14.234996 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:06:14.235278 (+ 282us) client_negotiation.cc:769] Sending connection context
0812 19:06:14.235451 (+ 173us) client_negotiation.cc:240] Negotiation successful
0812 19:06:14.235794 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8833,"thread_start_us":162,"threads_started":1}
I20260812 19:06:14.238175 9638 heartbeater.cc:347] Connected to a master server at 127.25.130.62:36757
I20260812 19:06:14.238430 9638 heartbeater.cc:464] Registering TS with master...
I20260812 19:06:14.238999 9638 heartbeater.cc:511] Master 127.25.130.62:36757 requested a full tablet report, sending...
W20260812 19:06:14.240536 9638 heartbeater.cc:644] Failed to heartbeat to 127.25.130.62:36757 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 19:06:14.240658 9638 heartbeater.cc:464] Registering TS with master...
I20260812 19:06:14.240830 9638 heartbeater.cc:511] Master 127.25.130.62:36757 requested a full tablet report, sending...
I20260812 19:06:14.241616 9638 heartbeater.cc:464] Registering TS with master...
I20260812 19:06:14.241786 9638 heartbeater.cc:511] Master 127.25.130.62:36757 requested a full tablet report, sending...
W20260812 19:06:14.242506 9638 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 19:06:14.382674 9641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:06:14.383076 9641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:06:14.383157 9641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:06:14.389540 9641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:06:14.389644 9641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:06:14.389734 9641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:06:14.395658 9641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:45981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.25.130.20:45319/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:37661
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=37813
--tserver_master_addrs=127.25.130.62:36757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:06:14.396847 9641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:06:14.397969 9641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:14.409647 9649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:14.409651 9648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:14.410279 9651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:14.411012 9641 server_base.cc:1034] running on GCE node
I20260812 19:06:14.411450 9641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:06:14.412169 9641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:06:14.413341 9641 hybrid_clock.cc:648] HybridClock initialized: now 1786561574413311 us; error 48 us; skew 500 ppm
Aug 12 19:06:14 dist-test-slave-49h3 krb5kdc[8086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561574, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:14.416218 9641 init.cc:377] Logged in from keytab as kudu/127.25.130.3@KRBTEST.COM (short username kudu)
I20260812 19:06:14.417387 9641 webserver.cc:460] Webserver started at http://127.25.130.3:37813/ using document root <none> and password file <none>
I20260812 19:06:14.418252 9641 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:14.418358 9641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 19:06:14 dist-test-slave-49h3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561574, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for HTTP/127.25.130.20@KRBTEST.COM
I20260812 19:06:14.443691 9641 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260812 19:06:14.446717 9661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:14.448585 9641 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 19:06:14.448752 9641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "0da1050c29b64f3d87984891059b70c6"
format_stamp: "Formatted at 2026-08-12 19:06:13 on dist-test-slave-49h3"
server_key: "66b5693dc02d5e545a4f937aaea1c419"
server_key_iv: "21f48aafe2005a7db01958228b85e204"
server_key_version: "kuduclusterkey@0"
I20260812 19:06:14.449544 9641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:14.479494 9641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:14.483234 9641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:06:14.483470 9641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:14.484297 9641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:06:14.485536 9641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:06:14.485672 9641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:14.485786 9641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:06:14.485858 9641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:14.498109 9641 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:37661
I20260812 19:06:14.498126 9805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:37661 every 8 connection(s)
I20260812 19:06:14.499486 9641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:06:14.505126 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 9641
Aug 12 19:06:14 dist-test-slave-49h3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561574, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:06:14.515493 9475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:14.501574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:33375 (local address 127.25.130.62:36757)
0812 19:06:14.501703 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:14.501707 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:14.502835 (+ 1128us) server_negotiation.cc:407] Connection header received
0812 19:06:14.503786 (+ 951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:14.503790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:14.503853 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:14.503929 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:14.505201 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:14.505981 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:14.507817 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:14.508134 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:14.508220 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:14.511260 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:14.511264 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:14.511266 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:14.511295 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:14.512870 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:14.513527 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:14.513532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:14.513534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:14.513648 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:14.514078 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:14.514082 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:14.514084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:14.514336 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:14.514439 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:14.515179 (+ 740us) server_negotiation.cc:299] Negotiation successful
0812 19:06:14.515311 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 19:06:14.516579 9808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:14.502082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:36757 (local address 127.25.130.3:33375)
0812 19:06:14.502675 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:14.502726 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:14.503574 (+ 848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:14.504137 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:14.504144 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:14.504424 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:06:14.505004 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:14.505038 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:14.506562 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:14.506564 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:14.507697 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:14.507707 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:14.508374 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:14.508377 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:14.508604 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:14.509617 (+ 1013us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:06:14.509639 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:06:14.511091 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:06:14.512984 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:14.512991 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:14.513009 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:14.513383 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:14.513751 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:14.513754 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:14.513757 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:14.513959 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:14.514500 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:06:14.514507 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:06:14.514867 (+ 360us) client_negotiation.cc:769] Sending connection context
0812 19:06:14.515090 (+ 223us) client_negotiation.cc:240] Negotiation successful
0812 19:06:14.515468 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":161,"threads_started":1}
I20260812 19:06:14.517937 9806 heartbeater.cc:347] Connected to a master server at 127.25.130.62:36757
I20260812 19:06:14.518227 9806 heartbeater.cc:464] Registering TS with master...
I20260812 19:06:14.518762 9806 heartbeater.cc:511] Master 127.25.130.62:36757 requested a full tablet report, sending...
I20260812 19:06:14.520267 9242 ts_manager.cc:194] Registered new tserver with Master: 0da1050c29b64f3d87984891059b70c6 (127.25.130.3:37661)
I20260812 19:06:14.521281 9810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:14.506232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32812 (local address 127.25.130.62:36757)
0812 19:06:14.507930 (+ 1698us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:14.507937 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:14.507948 (+ 11us) server_negotiation.cc:407] Connection header received
0812 19:06:14.507978 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:14.507980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:14.508036 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:14.508115 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:14.511239 (+ 3124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:14.512306 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:14.514084 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:14.514312 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:14.514399 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:14.516094 (+ 1695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:14.516097 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:14.516099 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:14.516122 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:14.519349 (+ 3227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:14.519855 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:14.519858 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:14.519860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:14.519951 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:14.520243 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:14.520246 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:14.520248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:14.520477 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:14.520689 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:14.520913 (+ 224us) server_negotiation.cc:299] Negotiation successful
0812 19:06:14.521052 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1626,"thread_start_us":155,"threads_started":1}
I20260812 19:06:14.522351 9242 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.3@KRBTEST.COM'} at 127.25.130.3:33375
Aug 12 19:06:14 dist-test-slave-49h3 krb5kdc[8086](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786561574, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:14.985375 9471 heartbeater.cc:464] Registering TS with master...
I20260812 19:06:14.985765 9471 heartbeater.cc:511] Master 127.25.130.62:36757 requested a full tablet report, sending...
I20260812 19:06:14.986788 9242 ts_manager.cc:194] Registered new tserver with Master: 0d712c5ebf36408bae8af256d49eb843 (127.25.130.1:36833)
I20260812 19:06:14.987582 9242 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.1@KRBTEST.COM'} at 127.25.130.1:50807
Aug 12 19:06:15 dist-test-slave-49h3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561574, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for HTTP/127.25.130.20@KRBTEST.COM
I20260812 19:06:15.242790 9638 heartbeater.cc:464] Registering TS with master...
I20260812 19:06:15.243122 9638 heartbeater.cc:511] Master 127.25.130.62:36757 requested a full tablet report, sending...
I20260812 19:06:15.244040 9242 ts_manager.cc:194] Registered new tserver with Master: e249e52fffd0421b91835b970bdbe5f0 (127.25.130.2:33931)
I20260812 19:06:15.244796 9242 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.2@KRBTEST.COM'} at 127.25.130.2:35745
I20260812 19:06:15.247848 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:06:15.277835 9820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:15.249605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32828 (local address 127.25.130.62:36757)
0812 19:06:15.257703 (+ 8098us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:15.257708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:15.257726 (+ 18us) server_negotiation.cc:407] Connection header received
0812 19:06:15.258327 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:15.258331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:15.258401 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:15.258493 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:15.259553 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.260654 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.270881 (+ 10227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.271281 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.271411 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:15.273520 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:15.273524 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:15.273527 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:15.273564 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:15.275370 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:15.275917 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:15.275921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:15.275923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:15.276027 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:15.276366 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:15.276370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:15.276371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:15.276620 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:15.276755 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:15.277488 (+ 733us) server_negotiation.cc:299] Negotiation successful
0812 19:06:15.277638 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7996,"thread_start_us":73,"threads_started":1}
I20260812 19:06:15.282672 9242 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32828:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:06:15.287890 9242 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:06:15.318275 9827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:15.307557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:33931 (local address 127.0.0.1:44344)
0812 19:06:15.309705 (+ 2148us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:15.309763 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:15.309899 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:15.310888 (+ 989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:15.310894 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:15.310919 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:06:15.311056 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:15.311064 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.312729 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.312733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:15.314075 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:15.314087 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.317706 (+ 3619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.317710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:15.317827 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:15.317867 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 19:06:15.317973 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 19:06:15.318109 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1959,"thread_start_us":137,"threads_started":1}
I20260812 19:06:15.326351 9830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:15.307727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44344 (local address 127.25.130.2:33931)
0812 19:06:15.310198 (+ 2471us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:15.310205 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:15.310229 (+ 24us) server_negotiation.cc:407] Connection header received
0812 19:06:15.310307 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:15.310313 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:15.310545 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:15.310699 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:06:15.311416 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.312593 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.316538 (+ 3945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.317351 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.317473 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:15.317636 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:15.326041 (+ 8405us) server_negotiation.cc:299] Negotiation successful
0812 19:06:15.326184 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2348,"thread_start_us":114,"threads_started":1}
I20260812 19:06:15.327118 9832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:15.307557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:37661 (local address 127.0.0.1:57392)
0812 19:06:15.314304 (+ 6747us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:15.314317 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:15.314397 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:15.315088 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:15.315091 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:15.315137 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:06:15.315228 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:15.315233 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.321316 (+ 6083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.321319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:15.322435 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:15.322447 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.326673 (+ 4226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.326677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:15.326778 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:15.326806 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 19:06:15.326880 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 19:06:15.326959 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6682,"mutex_wait_us":524,"thread_start_us":88,"threads_started":1}
I20260812 19:06:15.327692 9831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:15.307824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57392 (local address 127.25.130.3:37661)
0812 19:06:15.311347 (+ 3523us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:15.311353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:15.314495 (+ 3142us) server_negotiation.cc:407] Connection header received
0812 19:06:15.314581 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:15.314586 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:15.314807 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:15.314944 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:06:15.315348 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.316403 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.322588 (+ 6185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.323447 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.323555 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:15.323718 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:15.327431 (+ 3713us) server_negotiation.cc:299] Negotiation successful
0812 19:06:15.327560 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3409,"thread_start_us":124,"threads_started":1}
I20260812 19:06:15.328788 9834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:15.308957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33294 (local address 127.25.130.1:36833)
0812 19:06:15.309394 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:15.309402 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:15.319233 (+ 9831us) server_negotiation.cc:407] Connection header received
0812 19:06:15.319442 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:15.319448 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:15.319668 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:15.319786 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:06:15.320177 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.321196 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.324864 (+ 3668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.325692 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.325786 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:15.325926 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:15.328428 (+ 2502us) server_negotiation.cc:299] Negotiation successful
0812 19:06:15.328599 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":112,"threads_started":1}
I20260812 19:06:15.328323 9827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:15.308783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:36833 (local address 127.0.0.1:33294)
0812 19:06:15.319144 (+ 10361us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:15.319157 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:15.319309 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:15.319954 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:15.319957 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:15.319974 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:06:15.320045 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:15.320049 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.323778 (+ 3729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.323782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:15.324728 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:15.324737 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.327941 (+ 3204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.327945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:15.328029 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:15.328055 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 19:06:15.328140 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 19:06:15.328209 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10291,"thread_start_us":97,"threads_started":1}
I20260812 19:06:15.332731 9547 tablet_service.cc:1467] Processing CreateTablet for tablet 186114bbb65142f7a3264bcb16d83d6d (DEFAULT_TABLE table=test-table [id=be594ecd8f57450ea01e54b665a80b25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:06:15.334074 9547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 186114bbb65142f7a3264bcb16d83d6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:15.335631 9710 tablet_service.cc:1467] Processing CreateTablet for tablet 186114bbb65142f7a3264bcb16d83d6d (DEFAULT_TABLE table=test-table [id=be594ecd8f57450ea01e54b665a80b25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:06:15.335645 9369 tablet_service.cc:1467] Processing CreateTablet for tablet 186114bbb65142f7a3264bcb16d83d6d (DEFAULT_TABLE table=test-table [id=be594ecd8f57450ea01e54b665a80b25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:06:15.336679 9710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 186114bbb65142f7a3264bcb16d83d6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:15.336678 9369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 186114bbb65142f7a3264bcb16d83d6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:15.344398 9838 tablet_bootstrap.cc:492] T 186114bbb65142f7a3264bcb16d83d6d P 0d712c5ebf36408bae8af256d49eb843: Bootstrap starting.
I20260812 19:06:15.347209 9838 tablet_bootstrap.cc:654] T 186114bbb65142f7a3264bcb16d83d6d P 0d712c5ebf36408bae8af256d49eb843: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:15.347680 9836 tablet_bootstrap.cc:492] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6: Bootstrap starting.
I20260812 19:06:15.350056 9836 tablet_bootstrap.cc:654] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:15.351135 9838 log.cc:824] T 186114bbb65142f7a3264bcb16d83d6d P 0d712c5ebf36408bae8af256d49eb843: Log is configured to *not* fsync() on all Append() calls
I20260812 19:06:15.351636 9835 tablet_bootstrap.cc:492] T 186114bbb65142f7a3264bcb16d83d6d P e249e52fffd0421b91835b970bdbe5f0: Bootstrap starting.
I20260812 19:06:15.352993 9836 log.cc:824] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6: Log is configured to *not* fsync() on all Append() calls
I20260812 19:06:15.355472 9835 tablet_bootstrap.cc:654] T 186114bbb65142f7a3264bcb16d83d6d P e249e52fffd0421b91835b970bdbe5f0: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:15.356719 9835 log.cc:824] T 186114bbb65142f7a3264bcb16d83d6d P e249e52fffd0421b91835b970bdbe5f0: Log is configured to *not* fsync() on all Append() calls
I20260812 19:06:15.359254 9838 tablet_bootstrap.cc:492] T 186114bbb65142f7a3264bcb16d83d6d P 0d712c5ebf36408bae8af256d49eb843: No bootstrap required, opened a new log
I20260812 19:06:15.359543 9838 ts_tablet_manager.cc:1397] T 186114bbb65142f7a3264bcb16d83d6d P 0d712c5ebf36408bae8af256d49eb843: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.000s
I20260812 19:06:15.363976 9838 raft_consensus.cc:348] T 186114bbb65142f7a3264bcb16d83d6d P 0d712c5ebf36408bae8af256d49eb843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da1050c29b64f3d87984891059b70c6" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 37661 } } peers { permanent_uuid: "e249e52fffd0421b91835b970bdbe5f0" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33931 } } peers { permanent_uuid: "0d712c5ebf36408bae8af256d49eb843" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36833 } }
I20260812 19:06:15.366811 9836 tablet_bootstrap.cc:492] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6: No bootstrap required, opened a new log
I20260812 19:06:15.367132 9836 ts_tablet_manager.cc:1397] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.000s
I20260812 19:06:15.371644 9838 raft_consensus.cc:374] T 186114bbb65142f7a3264bcb16d83d6d P 0d712c5ebf36408bae8af256d49eb843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:15.371834 9838 raft_consensus.cc:729] T 186114bbb65142f7a3264bcb16d83d6d P 0d712c5ebf36408bae8af256d49eb843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d712c5ebf36408bae8af256d49eb843, State: Initialized, Role: FOLLOWER
I20260812 19:06:15.373101 9836 raft_consensus.cc:348] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da1050c29b64f3d87984891059b70c6" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 37661 } } peers { permanent_uuid: "e249e52fffd0421b91835b970bdbe5f0" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33931 } } peers { permanent_uuid: "0d712c5ebf36408bae8af256d49eb843" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36833 } }
I20260812 19:06:15.373519 9836 raft_consensus.cc:374] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:15.373598 9836 raft_consensus.cc:729] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0da1050c29b64f3d87984891059b70c6, State: Initialized, Role: FOLLOWER
I20260812 19:06:15.374065 9836 consensus_queue.cc:260] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da1050c29b64f3d87984891059b70c6" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 37661 } } peers { permanent_uuid: "e249e52fffd0421b91835b970bdbe5f0" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33931 } } peers { permanent_uuid: "0d712c5ebf36408bae8af256d49eb843" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36833 } }
I20260812 19:06:15.375249 9835 tablet_bootstrap.cc:492] T 186114bbb65142f7a3264bcb16d83d6d P e249e52fffd0421b91835b970bdbe5f0: No bootstrap required, opened a new log
I20260812 19:06:15.375310 9836 ts_tablet_manager.cc:1428] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260812 19:06:15.375545 9835 ts_tablet_manager.cc:1397] T 186114bbb65142f7a3264bcb16d83d6d P e249e52fffd0421b91835b970bdbe5f0: Time spent bootstrapping tablet: real 0.024s user 0.006s sys 0.000s
I20260812 19:06:15.375749 9806 heartbeater.cc:503] Master 127.25.130.62:36757 was elected leader, sending a full tablet report...
I20260812 19:06:15.375850 9838 consensus_queue.cc:260] T 186114bbb65142f7a3264bcb16d83d6d P 0d712c5ebf36408bae8af256d49eb843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da1050c29b64f3d87984891059b70c6" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 37661 } } peers { permanent_uuid: "e249e52fffd0421b91835b970bdbe5f0" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33931 } } peers { permanent_uuid: "0d712c5ebf36408bae8af256d49eb843" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36833 } }
I20260812 19:06:15.376993 9838 ts_tablet_manager.cc:1428] T 186114bbb65142f7a3264bcb16d83d6d P 0d712c5ebf36408bae8af256d49eb843: Time spent starting tablet: real 0.017s user 0.000s sys 0.006s
I20260812 19:06:15.377379 9471 heartbeater.cc:503] Master 127.25.130.62:36757 was elected leader, sending a full tablet report...
I20260812 19:06:15.381253 9835 raft_consensus.cc:348] T 186114bbb65142f7a3264bcb16d83d6d P e249e52fffd0421b91835b970bdbe5f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da1050c29b64f3d87984891059b70c6" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 37661 } } peers { permanent_uuid: "e249e52fffd0421b91835b970bdbe5f0" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33931 } } peers { permanent_uuid: "0d712c5ebf36408bae8af256d49eb843" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36833 } }
I20260812 19:06:15.381744 9835 raft_consensus.cc:374] T 186114bbb65142f7a3264bcb16d83d6d P e249e52fffd0421b91835b970bdbe5f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:15.381944 9835 raft_consensus.cc:729] T 186114bbb65142f7a3264bcb16d83d6d P e249e52fffd0421b91835b970bdbe5f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e249e52fffd0421b91835b970bdbe5f0, State: Initialized, Role: FOLLOWER
I20260812 19:06:15.382566 9835 consensus_queue.cc:260] T 186114bbb65142f7a3264bcb16d83d6d P e249e52fffd0421b91835b970bdbe5f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da1050c29b64f3d87984891059b70c6" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 37661 } } peers { permanent_uuid: "e249e52fffd0421b91835b970bdbe5f0" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33931 } } peers { permanent_uuid: "0d712c5ebf36408bae8af256d49eb843" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36833 } }
I20260812 19:06:15.383826 9835 ts_tablet_manager.cc:1428] T 186114bbb65142f7a3264bcb16d83d6d P e249e52fffd0421b91835b970bdbe5f0: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260812 19:06:15.384052 9638 heartbeater.cc:503] Master 127.25.130.62:36757 was elected leader, sending a full tablet report...
W20260812 19:06:15.450105 9472 tablet.cc:2369] T 186114bbb65142f7a3264bcb16d83d6d P 0d712c5ebf36408bae8af256d49eb843: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 19:06:15.459093 9639 tablet.cc:2369] T 186114bbb65142f7a3264bcb16d83d6d P e249e52fffd0421b91835b970bdbe5f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 19:06:15.501219 9807 tablet.cc:2369] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 19:06:15 dist-test-slave-49h3 krb5kdc[8086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561574, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for HTTP/127.25.130.20@KRBTEST.COM
I20260812 19:06:15.671144 9843 raft_consensus.cc:482] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:15.671444 9843 raft_consensus.cc:504] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da1050c29b64f3d87984891059b70c6" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 37661 } } peers { permanent_uuid: "e249e52fffd0421b91835b970bdbe5f0" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33931 } } peers { permanent_uuid: "0d712c5ebf36408bae8af256d49eb843" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36833 } }
I20260812 19:06:15.672959 9843 leader_election.cc:288] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e249e52fffd0421b91835b970bdbe5f0 (127.25.130.2:33931), 0d712c5ebf36408bae8af256d49eb843 (127.25.130.1:36833)
I20260812 19:06:15.678131 9845 raft_consensus.cc:482] T 186114bbb65142f7a3264bcb16d83d6d P e249e52fffd0421b91835b970bdbe5f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:15.678344 9845 raft_consensus.cc:504] T 186114bbb65142f7a3264bcb16d83d6d P e249e52fffd0421b91835b970bdbe5f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da1050c29b64f3d87984891059b70c6" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 37661 } } peers { permanent_uuid: "e249e52fffd0421b91835b970bdbe5f0" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33931 } } peers { permanent_uuid: "0d712c5ebf36408bae8af256d49eb843" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36833 } }
I20260812 19:06:15.679557 9845 leader_election.cc:288] T 186114bbb65142f7a3264bcb16d83d6d P e249e52fffd0421b91835b970bdbe5f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0da1050c29b64f3d87984891059b70c6 (127.25.130.3:37661), 0d712c5ebf36408bae8af256d49eb843 (127.25.130.1:36833)
I20260812 19:06:15.682029 9830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:15.674465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:34401 (local address 127.25.130.2:33931)
0812 19:06:15.674592 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:15.674596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:15.675143 (+ 547us) server_negotiation.cc:407] Connection header received
0812 19:06:15.675379 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:15.675383 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:15.675436 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:15.675486 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:06:15.675902 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.676686 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.678850 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.679469 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.679572 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:15.679629 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:15.681804 (+ 2175us) server_negotiation.cc:299] Negotiation successful
0812 19:06:15.681901 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 19:06:15.682652 9849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:15.673495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:33931 (local address 127.25.130.3:34401)
0812 19:06:15.675045 (+ 1550us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:15.675061 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:15.675236 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:15.675638 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:15.675641 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:15.675661 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:06:15.675776 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:15.675783 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.676872 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.676875 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:15.678679 (+ 1804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:15.678691 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.681522 (+ 2831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.681525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:15.681620 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:15.681649 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 19:06:15.682397 (+ 748us) client_negotiation.cc:240] Negotiation successful
0812 19:06:15.682481 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1470,"thread_start_us":127,"threads_started":1}
I20260812 19:06:15.683449 9582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "186114bbb65142f7a3264bcb16d83d6d" candidate_uuid: "0da1050c29b64f3d87984891059b70c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e249e52fffd0421b91835b970bdbe5f0" is_pre_election: true
I20260812 19:06:15.683717 9582 raft_consensus.cc:2454] T 186114bbb65142f7a3264bcb16d83d6d P e249e52fffd0421b91835b970bdbe5f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0da1050c29b64f3d87984891059b70c6 in term 0.
I20260812 19:06:15.684319 9664 leader_election.cc:302] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da1050c29b64f3d87984891059b70c6, e249e52fffd0421b91835b970bdbe5f0; no voters:
I20260812 19:06:15.684638 9843 raft_consensus.cc:2781] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:15.684861 9843 raft_consensus.cc:482] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:15.684923 9843 raft_consensus.cc:3037] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:15.686224 9834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:15.674061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:34703 (local address 127.25.130.1:36833)
0812 19:06:15.674226 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:15.674231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:15.681761 (+ 7530us) server_negotiation.cc:407] Connection header received
0812 19:06:15.681955 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:15.681959 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:15.682007 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:15.682054 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:06:15.682487 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.683161 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.684757 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.685322 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.685418 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:15.685523 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:15.685955 (+ 432us) server_negotiation.cc:299] Negotiation successful
0812 19:06:15.686087 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 19:06:15.686491 9843 raft_consensus.cc:504] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da1050c29b64f3d87984891059b70c6" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 37661 } } peers { permanent_uuid: "e249e52fffd0421b91835b970bdbe5f0" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33931 } } peers { permanent_uuid: "0d712c5ebf36408bae8af256d49eb843" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36833 } }
I20260812 19:06:15.686882 9843 leader_election.cc:288] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6 [CANDIDATE]: Term 1 election: Requested vote from peers e249e52fffd0421b91835b970bdbe5f0 (127.25.130.2:33931), 0d712c5ebf36408bae8af256d49eb843 (127.25.130.1:36833)
I20260812 19:06:15.686956 9851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:15.673952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:36833 (local address 127.25.130.3:34703)
0812 19:06:15.681670 (+ 7718us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:15.681684 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:15.681842 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:15.682220 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:15.682223 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:15.682242 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:06:15.682347 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:15.682353 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.683313 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.683316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:15.684606 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:15.684616 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.685701 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.685704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:15.685805 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:15.685830 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 19:06:15.686720 (+ 890us) client_negotiation.cc:240] Negotiation successful
0812 19:06:15.686799 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7648,"thread_start_us":96,"threads_started":1}
I20260812 19:06:15.689365 9410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "186114bbb65142f7a3264bcb16d83d6d" candidate_uuid: "0da1050c29b64f3d87984891059b70c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d712c5ebf36408bae8af256d49eb843" is_pre_election: true
I20260812 19:06:15.689760 9410 raft_consensus.cc:2454] T 186114bbb65142f7a3264bcb16d83d6d P 0d712c5ebf36408bae8af256d49eb843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0da1050c29b64f3d87984891059b70c6 in term 0.
I20260812 19:06:15.690122 9409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "186114bbb65142f7a3264bcb16d83d6d" candidate_uuid: "0da1050c29b64f3d87984891059b70c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d712c5ebf36408bae8af256d49eb843"
I20260812 19:06:15.690233 9409 raft_consensus.cc:3037] T 186114bbb65142f7a3264bcb16d83d6d P 0d712c5ebf36408bae8af256d49eb843 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:15.691830 9409 raft_consensus.cc:2454] T 186114bbb65142f7a3264bcb16d83d6d P 0d712c5ebf36408bae8af256d49eb843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0da1050c29b64f3d87984891059b70c6 in term 1.
I20260812 19:06:15.692615 9582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "186114bbb65142f7a3264bcb16d83d6d" candidate_uuid: "0da1050c29b64f3d87984891059b70c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e249e52fffd0421b91835b970bdbe5f0"
I20260812 19:06:15.692776 9582 raft_consensus.cc:3037] T 186114bbb65142f7a3264bcb16d83d6d P e249e52fffd0421b91835b970bdbe5f0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:15.694020 9664 leader_election.cc:302] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d712c5ebf36408bae8af256d49eb843, 0da1050c29b64f3d87984891059b70c6; no voters:
I20260812 19:06:15.695127 9582 raft_consensus.cc:2454] T 186114bbb65142f7a3264bcb16d83d6d P e249e52fffd0421b91835b970bdbe5f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0da1050c29b64f3d87984891059b70c6 in term 1.
I20260812 19:06:15.695683 9843 raft_consensus.cc:2781] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:15.695959 9843 raft_consensus.cc:686] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6 [term 1 LEADER]: Becoming Leader. State: Replica: 0da1050c29b64f3d87984891059b70c6, State: Running, Role: LEADER
I20260812 19:06:15.696358 9843 consensus_queue.cc:237] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da1050c29b64f3d87984891059b70c6" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 37661 } } peers { permanent_uuid: "e249e52fffd0421b91835b970bdbe5f0" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33931 } } peers { permanent_uuid: "0d712c5ebf36408bae8af256d49eb843" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36833 } }
I20260812 19:06:15.700112 9242 catalog_manager.cc:5495] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0da1050c29b64f3d87984891059b70c6 (127.25.130.3). New cstate: current_term: 1 leader_uuid: "0da1050c29b64f3d87984891059b70c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da1050c29b64f3d87984891059b70c6" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 37661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e249e52fffd0421b91835b970bdbe5f0" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d712c5ebf36408bae8af256d49eb843" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36833 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:15.701965 9831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:15.680946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:41623 (local address 127.25.130.3:37661)
0812 19:06:15.681393 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:15.681398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:15.697772 (+ 16374us) server_negotiation.cc:407] Connection header received
0812 19:06:15.697970 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:15.697973 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:15.698020 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:15.698063 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:06:15.698504 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.699182 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.700677 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.701224 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.701319 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:15.701373 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:15.701753 (+ 380us) server_negotiation.cc:299] Negotiation successful
0812 19:06:15.701837 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356}
I20260812 19:06:15.702551 9853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:15.679891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:37661 (local address 127.25.130.2:41623)
0812 19:06:15.697648 (+ 17757us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:15.697665 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:15.697854 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:15.698200 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:15.698203 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:15.698233 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:06:15.698373 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:15.698380 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.699343 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.699346 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:15.700536 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:15.700548 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.701502 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.701505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:15.701607 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:15.701634 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 19:06:15.702302 (+ 668us) client_negotiation.cc:240] Negotiation successful
0812 19:06:15.702384 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17666,"thread_start_us":152,"threads_started":1}
I20260812 19:06:15.703274 9719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "186114bbb65142f7a3264bcb16d83d6d" candidate_uuid: "e249e52fffd0421b91835b970bdbe5f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0da1050c29b64f3d87984891059b70c6" is_pre_election: true
I20260812 19:06:15.709827 9834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:15.681086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:49317 (local address 127.25.130.1:36833)
0812 19:06:15.686402 (+ 5316us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:15.686407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:15.701980 (+ 15573us) server_negotiation.cc:407] Connection header received
0812 19:06:15.702185 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:15.702188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:15.702241 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:15.702286 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:06:15.703998 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.704729 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.708312 (+ 3583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.708953 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.709059 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:15.709127 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:15.709553 (+ 426us) server_negotiation.cc:299] Negotiation successful
0812 19:06:15.709644 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5254,"thread_start_us":120,"threads_started":1}
I20260812 19:06:15.710716 9854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:15.680361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:36833 (local address 127.25.130.2:49317)
0812 19:06:15.701888 (+ 21527us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:15.701903 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:15.702057 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:15.703695 (+ 1638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:15.703698 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:15.703730 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:06:15.703859 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:15.703866 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.704872 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.704875 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:15.706133 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:15.706144 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.709233 (+ 3089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.709237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:15.709355 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:15.709380 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 19:06:15.710464 (+ 1084us) client_negotiation.cc:240] Negotiation successful
0812 19:06:15.710553 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21458,"thread_start_us":101,"threads_started":1}
I20260812 19:06:15.711350 9409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "186114bbb65142f7a3264bcb16d83d6d" candidate_uuid: "e249e52fffd0421b91835b970bdbe5f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d712c5ebf36408bae8af256d49eb843" is_pre_election: true
I20260812 19:06:15.711618 9409 raft_consensus.cc:2379] T 186114bbb65142f7a3264bcb16d83d6d P 0d712c5ebf36408bae8af256d49eb843 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e249e52fffd0421b91835b970bdbe5f0 in current term 1: Already voted for candidate 0da1050c29b64f3d87984891059b70c6 in this term.
I20260812 19:06:15.712314 9500 leader_election.cc:302] T 186114bbb65142f7a3264bcb16d83d6d P e249e52fffd0421b91835b970bdbe5f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e249e52fffd0421b91835b970bdbe5f0; no voters: 0d712c5ebf36408bae8af256d49eb843, 0da1050c29b64f3d87984891059b70c6
I20260812 19:06:15.712909 9845 raft_consensus.cc:2726] T 186114bbb65142f7a3264bcb16d83d6d P e249e52fffd0421b91835b970bdbe5f0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:15.788121 9831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:15.782055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42858 (local address 127.25.130.3:37661)
0812 19:06:15.782183 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:15.782188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:15.782510 (+ 322us) server_negotiation.cc:407] Connection header received
0812 19:06:15.782752 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:15.782755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:15.782827 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:15.782906 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:06:15.783293 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.784089 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.785716 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:15.786113 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:15.786201 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:15.786683 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:06:15.787423 (+ 740us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:06:15.787562 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:15.787741 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 19:06:15.787818 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 19:06:15.797312 9409 raft_consensus.cc:1261] T 186114bbb65142f7a3264bcb16d83d6d P 0d712c5ebf36408bae8af256d49eb843 [term 1 FOLLOWER]: Refusing update from remote peer 0da1050c29b64f3d87984891059b70c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:15.797367 9582 raft_consensus.cc:1261] T 186114bbb65142f7a3264bcb16d83d6d P e249e52fffd0421b91835b970bdbe5f0 [term 1 FOLLOWER]: Refusing update from remote peer 0da1050c29b64f3d87984891059b70c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:15.798245 9857 consensus_queue.cc:1035] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e249e52fffd0421b91835b970bdbe5f0" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 33931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:15.798488 9843 consensus_queue.cc:1035] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d712c5ebf36408bae8af256d49eb843" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 36833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:15.813023 9861 mvcc.cc:205] Tried to move back new op lower bound from 7317756214455152640 to 7317756214056337408. Current Snapshot: MvccSnapshot[applied={T|T < 7317756214455152640}]
I20260812 19:06:15.818184 9864 mvcc.cc:205] Tried to move back new op lower bound from 7317756214455152640 to 7317756214056337408. Current Snapshot: MvccSnapshot[applied={T|T < 7317756214455152640}]
I20260812 19:06:15.820880 9862 mvcc.cc:205] Tried to move back new op lower bound from 7317756214455152640 to 7317756214056337408. Current Snapshot: MvccSnapshot[applied={T|T < 7317756214455152640}]
I20260812 19:06:15.828553 9242 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:06:15.828753 9242 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:06:15.832057 9242 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cd5f9ca68867402196c0be762560c4ec: Sending DeleteTablet for 3 replicas of tablet 186114bbb65142f7a3264bcb16d83d6d
I20260812 19:06:15.833163 9710 tablet_service.cc:1514] Processing DeleteTablet for tablet 186114bbb65142f7a3264bcb16d83d6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:06:15 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:57392
I20260812 19:06:15.833390 9547 tablet_service.cc:1514] Processing DeleteTablet for tablet 186114bbb65142f7a3264bcb16d83d6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:06:15 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:44344
I20260812 19:06:15.833763 9369 tablet_service.cc:1514] Processing DeleteTablet for tablet 186114bbb65142f7a3264bcb16d83d6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:06:15 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:33294
I20260812 19:06:15.834293 9878 tablet_replica.cc:330] T 186114bbb65142f7a3264bcb16d83d6d P 0d712c5ebf36408bae8af256d49eb843: stopping tablet replica
I20260812 19:06:15.834649 9878 raft_consensus.cc:2229] T 186114bbb65142f7a3264bcb16d83d6d P 0d712c5ebf36408bae8af256d49eb843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:15.834823 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 9284
W20260812 19:06:15.845563 9225 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.130.1:36833 (error 108)
I20260812 19:06:15.845705 9877 tablet_replica.cc:330] T 186114bbb65142f7a3264bcb16d83d6d P e249e52fffd0421b91835b970bdbe5f0: stopping tablet replica
I20260812 19:06:15.846094 9877 raft_consensus.cc:2229] T 186114bbb65142f7a3264bcb16d83d6d P e249e52fffd0421b91835b970bdbe5f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:15.846470 9877 raft_consensus.cc:2258] T 186114bbb65142f7a3264bcb16d83d6d P e249e52fffd0421b91835b970bdbe5f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:15.846922 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 9473
I20260812 19:06:15.846927 9879 tablet_replica.cc:330] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6: stopping tablet replica
I20260812 19:06:15.847254 9879 raft_consensus.cc:2229] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6 [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:06:15.847687 9879 raft_consensus.cc:2258] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:15.855311 9879 ts_tablet_manager.cc:1905] T 186114bbb65142f7a3264bcb16d83d6d P 0da1050c29b64f3d87984891059b70c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:06:15.855862 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 9641
I20260812 19:06:15.860630 9882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:15.854191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:36833 (local address 127.0.0.1:43758)
0812 19:06:15.860407 (+ 6216us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:15.860504 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.1:36833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6115,"thread_start_us":69,"threads_started":1}
W20260812 19:06:15.861378 9225 catalog_manager.cc:4584] TS 0d712c5ebf36408bae8af256d49eb843 (127.25.130.1:36833): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 186114bbb65142f7a3264bcb16d83d6d: Network error: Client connection negotiation failed: client connection to 127.25.130.1:36833: connect: Connection refused (error 111)
I20260812 19:06:15.861123 9884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:15.860324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:33931 (local address 127.0.0.1:57504)
0812 19:06:15.860961 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:15.861019 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.2:33931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":427,"threads_started":1}
I20260812 19:06:15.866940 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 9202
I20260812 19:06:15.879408 26120 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 19:06:16.224617 26120 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 19:06:16.224736 26120 mini_ranger.cc:66] Stopping Ranger...
I20260812 19:06:16.580235 26120 mini_ranger.cc:68] Stopped Ranger
2026-08-12 19:06:16.580 UTC [8122] LOG: received smart shutdown request
2026-08-12 19:06:16.584 UTC [8122] LOG: background worker "logical replication launcher" (PID 8131) exited with exit code 1
2026-08-12 19:06:16.586 UTC [8126] LOG: shutting down
2026-08-12 19:06:16.747 UTC [8122] LOG: database system is shut down
2026-08-12T19:06:16Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48879 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:06:16 dist-test-slave-49h3 krb5kdc[9896](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:06:16 dist-test-slave-49h3 krb5kdc[9896](info): set up 2 sockets
Aug 12 19:06:16 dist-test-slave-49h3 krb5kdc[9896](info): commencing operation
krb5kdc: starting...
W20260812 19:06:17.932344 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:06:18 dist-test-slave-49h3 krb5kdc[9896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:06:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:06:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:06:18.062254 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:37403
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:39105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:37403,127.25.130.61:33851,127.25.130.60:39377
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:06:18.176715 9912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:06:18.177023 9912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:06:18.177095 9912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:06:18.181154 9912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:06:18.181222 9912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:06:18.181249 9912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:06:18.181272 9912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:06:18.181298 9912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:06:18.186827 9912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:39105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:37403,127.25.130.61:33851,127.25.130.60:39377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:37403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:06:18.187973 9912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:06:18.189014 9912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:18.197319 9917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:18.197393 9912 server_base.cc:1034] running on GCE node
W20260812 19:06:18.197314 9921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:18.197314 9918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:18.198230 9912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:06:18.199375 9912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:06:18.200557 9912 hybrid_clock.cc:648] HybridClock initialized: now 1786561578200545 us; error 31 us; skew 500 ppm
Aug 12 19:06:18 dist-test-slave-49h3 krb5kdc[9896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561578, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:18.206530 9912 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:06:18.207542 9912 webserver.cc:460] Webserver started at http://127.25.130.62:36335/ using document root <none> and password file <none>
I20260812 19:06:18.208110 9912 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:18.208181 9912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:18.208398 9912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:06:18.210583 9912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "fb494dfef55d46d384f4359f46e8a813"
format_stamp: "Formatted at 2026-08-12 19:06:18 on dist-test-slave-49h3"
server_key: "c93acccba76ccdfdde6fd0ffda8fe767"
server_key_iv: "91057aaf47b0910d5e7576407c705df7"
server_key_version: "encryptionkey@0"
I20260812 19:06:18.211164 9912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "fb494dfef55d46d384f4359f46e8a813"
format_stamp: "Formatted at 2026-08-12 19:06:18 on dist-test-slave-49h3"
server_key: "c93acccba76ccdfdde6fd0ffda8fe767"
server_key_iv: "91057aaf47b0910d5e7576407c705df7"
server_key_version: "encryptionkey@0"
I20260812 19:06:18.215039 9912 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260812 19:06:18.217654 9931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:18.219223 9912 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 19:06:18.219403 9912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "fb494dfef55d46d384f4359f46e8a813"
format_stamp: "Formatted at 2026-08-12 19:06:18 on dist-test-slave-49h3"
server_key: "c93acccba76ccdfdde6fd0ffda8fe767"
server_key_iv: "91057aaf47b0910d5e7576407c705df7"
server_key_version: "encryptionkey@0"
I20260812 19:06:18.219545 9912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:18.237833 9912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:18.241096 9912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:06:18.241267 9912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:18.248631 9912 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:37403
I20260812 19:06:18.248647 9989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:37403 every 8 connection(s)
I20260812 19:06:18.249681 9912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:06:18.252629 9990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.256606 9990 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37403 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:06:18.259865 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 9912
I20260812 19:06:18.260015 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:06:18.260317 26120 external_mini_cluster.cc:1428] Setting key e310e6e18d46e7d7f445fad5f0a5cd4d
Aug 12 19:06:18 dist-test-slave-49h3 krb5kdc[9896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561578, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.130.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.61@KRBTEST.COM" created.
I20260812 19:06:18.273711 9993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.259055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47158 (local address 127.25.130.62:37403)
0812 19:06:18.259526 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:18.259546 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:18.259843 (+ 297us) server_negotiation.cc:407] Connection header received
0812 19:06:18.261106 (+ 1263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:18.261111 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:18.261300 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:18.261490 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:18.262570 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.263908 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.265338 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.265805 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.265967 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:18.268803 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:18.268809 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:18.268821 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:18.268844 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:18.270564 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.271068 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.271074 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.271076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.271198 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.271488 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.271490 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.271491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.271912 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:18.272116 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:18.272462 (+ 346us) server_negotiation.cc:299] Negotiation successful
0812 19:06:18.272685 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":156,"threads_started":1}
I20260812 19:06:18.273710 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.259036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37403 (local address 127.0.0.1:47158)
0812 19:06:18.259664 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:18.259722 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:18.260799 (+ 1077us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:18.261525 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:18.261530 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:18.262008 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:06:18.262382 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:18.262405 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.264072 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.264077 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:18.265190 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:18.265203 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.266005 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.266021 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:18.266161 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:18.266773 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:06:18.266796 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:06:18.268615 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:06:18.270686 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:18.270696 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:18.270705 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:18.270960 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:18.271290 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:18.271293 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:18.271294 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:18.271418 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:18.272068 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:06:18.272074 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:06:18.272218 (+ 144us) client_negotiation.cc:769] Sending connection context
0812 19:06:18.272366 (+ 148us) client_negotiation.cc:240] Negotiation successful
0812 19:06:18.272806 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"mutex_wait_us":97,"spinlock_wait_cycles":36736,"thread_start_us":174,"threads_started":1}
I20260812 19:06:18.276764 9990 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.61" port: 33851 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:06:18.277616 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.277264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:33851 (local address 127.0.0.1:60324)
0812 19:06:18.277419 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:18.277542 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.61:33851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260812 19:06:18.277859 9932 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.130.61:33851: connect: Connection refused (error 111)
I20260812 19:06:18.278751 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.278545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:33851 (local address 127.0.0.1:60340)
0812 19:06:18.278649 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:18.278695 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.61:33851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 19:06:18.278904 9990 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.130.61:33851: Network error: Client connection negotiation failed: client connection to 127.25.130.61:33851: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.61.keytab.
Entry for principal kudu/127.25.130.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.61.keytab.
I20260812 19:06:18.290012 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.61:33851
--webserver_interface=127.25.130.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.61.keytab
--principal=kudu/127.25.130.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:39105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:37403,127.25.130.61:33851,127.25.130.60:39377
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:06:18.328148 9990 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.130.61" port: 33851 } attempt: 1
I20260812 19:06:18.329090 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.328745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:33851 (local address 127.0.0.1:60352)
0812 19:06:18.328911 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:18.328977 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.61:33851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 19:06:18.329885 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.329667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:33851 (local address 127.0.0.1:60358)
0812 19:06:18.329777 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:18.329824 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.61:33851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 19:06:18.330004 9990 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.130.61:33851: Network error: Client connection negotiation failed: client connection to 127.25.130.61:33851: connect: Connection refused (error 111)
I20260812 19:06:18.398190 9990 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.130.61" port: 33851 } attempt: 2
I20260812 19:06:18.399156 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.398783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:33851 (local address 127.0.0.1:60374)
0812 19:06:18.398963 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:18.399059 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.61:33851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 19:06:18.399945 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.399709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:33851 (local address 127.0.0.1:60376)
0812 19:06:18.399825 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:18.399875 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.61:33851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 19:06:18.400079 9990 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.130.61:33851: Network error: Client connection negotiation failed: client connection to 127.25.130.61:33851: connect: Connection refused (error 111)
W20260812 19:06:18.406221 9998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:06:18.406517 9998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:06:18.406569 9998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:06:18.410764 9998 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:06:18.410838 9998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:06:18.410856 9998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:06:18.410871 9998 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:06:18.410889 9998 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:06:18.416509 9998 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:39105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:37403,127.25.130.61:33851,127.25.130.60:39377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.61.keytab
--principal=kudu/127.25.130.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.61:33851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.25.130.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:06:18.417750 9998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:06:18.418895 9998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:18.427783 10004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:18.427783 10010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:18.427784 10005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:18.427924 9998 server_base.cc:1034] running on GCE node
I20260812 19:06:18.428715 9998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:06:18.429962 9998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:06:18.431175 9998 hybrid_clock.cc:648] HybridClock initialized: now 1786561578431146 us; error 41 us; skew 500 ppm
Aug 12 19:06:18 dist-test-slave-49h3 krb5kdc[9896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561578, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:18.434188 9998 init.cc:377] Logged in from keytab as kudu/127.25.130.61@KRBTEST.COM (short username kudu)
I20260812 19:06:18.435175 9998 webserver.cc:460] Webserver started at http://127.25.130.61:43377/ using document root <none> and password file <none>
I20260812 19:06:18.435729 9998 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:18.435792 9998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:18.435974 9998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:06:18.438176 9998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-1/data/instance:
uuid: "c76b7dd5bfaa45d499a4fa0219a11e45"
format_stamp: "Formatted at 2026-08-12 19:06:18 on dist-test-slave-49h3"
server_key: "f64394e8eb4a53d7c7a736a8623a7a5c"
server_key_iv: "6224599bdb588ea6070c3466679b9851"
server_key_version: "encryptionkey@0"
I20260812 19:06:18.438740 9998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-1/wal/instance:
uuid: "c76b7dd5bfaa45d499a4fa0219a11e45"
format_stamp: "Formatted at 2026-08-12 19:06:18 on dist-test-slave-49h3"
server_key: "f64394e8eb4a53d7c7a736a8623a7a5c"
server_key_iv: "6224599bdb588ea6070c3466679b9851"
server_key_version: "encryptionkey@0"
I20260812 19:06:18.442692 9998 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:06:18.445556 10018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:18.446871 9998 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:06:18.447002 9998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-1/wal
uuid: "c76b7dd5bfaa45d499a4fa0219a11e45"
format_stamp: "Formatted at 2026-08-12 19:06:18 on dist-test-slave-49h3"
server_key: "f64394e8eb4a53d7c7a736a8623a7a5c"
server_key_iv: "6224599bdb588ea6070c3466679b9851"
server_key_version: "encryptionkey@0"
I20260812 19:06:18.447122 9998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:18.476433 9998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:18.479370 9998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:06:18.479557 9998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:18.486524 9998 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.61:33851
I20260812 19:06:18.486547 10076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.61:33851 every 8 connection(s)
I20260812 19:06:18.487515 9998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-1/data/info.pb
I20260812 19:06:18.487969 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 9998
I20260812 19:06:18.488091 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-1/wal/instance
I20260812 19:06:18.488353 26120 external_mini_cluster.cc:1428] Setting key dc69bec2c16079fded8d1c8248105076
I20260812 19:06:18.490561 10077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.491231 9990 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.130.61" port: 33851 } attempt: 3
I20260812 19:06:18.496109 10077 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37403 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.25.130.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.60@KRBTEST.COM" created.
Aug 12 19:06:18 dist-test-slave-49h3 krb5kdc[9896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561578, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for kudu/127.25.130.61@KRBTEST.COM
Aug 12 19:06:18 dist-test-slave-49h3 krb5kdc[9896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561578, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.61@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:06:18.506309 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.491748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:33851 (local address 127.0.0.1:60380)
0812 19:06:18.492013 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:18.492044 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:18.492144 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:18.494858 (+ 2714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:18.494861 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:18.495071 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:06:18.495220 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:18.495230 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.496462 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.496465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:18.497049 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:18.497060 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.497821 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.497825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:18.497933 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:18.498555 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:06:18.498570 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:06:18.501787 (+ 3217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:06:18.504251 (+ 2464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:18.504255 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:18.504257 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:18.504448 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:18.504841 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:18.504844 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:18.504846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:18.504933 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:18.505723 (+ 790us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:06:18.505727 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:06:18.505834 (+ 107us) client_negotiation.cc:769] Sending connection context
0812 19:06:18.505939 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 19:06:18.506129 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172}
I20260812 19:06:18.508266 10083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.492483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60380 (local address 127.25.130.61:33851)
0812 19:06:18.493069 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:18.493084 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:18.493122 (+ 38us) server_negotiation.cc:407] Connection header received
0812 19:06:18.494208 (+ 1086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:18.494228 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:18.494574 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:18.494888 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:18.495339 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.496329 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.497172 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.497694 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.497850 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:18.501962 (+ 4112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:18.501967 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:18.501976 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:18.502004 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:18.504136 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.504605 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.504610 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.504616 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.504747 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.505016 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.505019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.505021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.505608 (+ 587us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:18.505765 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:18.506789 (+ 1024us) server_negotiation.cc:299] Negotiation successful
0812 19:06:18.507134 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":155,"threads_started":1}
I20260812 19:06:18.508361 9993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.498345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47160 (local address 127.25.130.62:37403)
0812 19:06:18.498471 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:18.498475 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:18.498961 (+ 486us) server_negotiation.cc:407] Connection header received
0812 19:06:18.499109 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:18.499113 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:18.499166 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:18.499242 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:18.499868 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.500375 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.501325 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.502337 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.502437 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:18.505132 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:18.505136 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:18.505138 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:18.505161 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:18.506743 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.507167 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.507171 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.507173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.507259 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.507502 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.507505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.507507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.507673 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:18.507759 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:18.508097 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 19:06:18.508215 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 19:06:18.508863 10085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.498338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37403 (local address 127.0.0.1:47160)
0812 19:06:18.498808 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:18.498861 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:18.498993 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:18.499301 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:18.499307 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:18.499645 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:06:18.499776 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:18.499782 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.500488 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.500491 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:18.501217 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:18.501227 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.502541 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.502545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:18.502677 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:18.503180 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:06:18.503211 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:06:18.504940 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:06:18.506843 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:18.506853 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:18.506859 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:18.507076 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:18.507338 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:18.507342 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:18.507344 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:18.507424 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:18.507775 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:06:18.507782 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:06:18.507892 (+ 110us) client_negotiation.cc:769] Sending connection context
0812 19:06:18.508012 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 19:06:18.508205 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":169,"threads_started":1}
I20260812 19:06:18.510001 10077 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.61" port: 33851 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:06:18.510404 9990 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:06:18.511032 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.510793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:39377 (local address 127.0.0.1:38654)
0812 19:06:18.510907 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:18.510960 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:39377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 19:06:18.511741 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.511501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:39377 (local address 127.0.0.1:38668)
0812 19:06:18.511638 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:18.511686 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:39377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260812 19:06:18.511897 9990 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.130.60:39377: Network error: Client connection negotiation failed: client connection to 127.25.130.60:39377: connect: Connection refused (error 111)
Aug 12 19:06:18 dist-test-slave-49h3 krb5kdc[9896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561578, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.61@KRBTEST.COM for kudu/127.25.130.61@KRBTEST.COM
I20260812 19:06:18.518451 10085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.510574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:33851 (local address 127.0.0.1:60394)
0812 19:06:18.510737 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:18.510749 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:18.510850 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:18.511114 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:18.511117 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:18.511283 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:06:18.511493 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:18.511501 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.512508 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.512511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:18.513040 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:18.513048 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.513595 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.513597 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:18.513684 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:18.514117 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:06:18.514128 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:06:18.515423 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:06:18.517104 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:18.517106 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:18.517107 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:18.517283 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:18.517580 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:18.517582 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:18.517584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:18.517658 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:18.518006 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:06:18.518008 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:06:18.518094 (+ 86us) client_negotiation.cc:769] Sending connection context
0812 19:06:18.518187 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 19:06:18.518297 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":13,"spinlock_wait_cycles":199936}
I20260812 19:06:18.518512 10083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.510623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60394 (local address 127.25.130.61:33851)
0812 19:06:18.510703 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:18.510707 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:18.510792 (+ 85us) server_negotiation.cc:407] Connection header received
0812 19:06:18.510957 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:18.510960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:18.511006 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:18.511089 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:18.511615 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.512387 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.513171 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.513506 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.513593 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:18.515553 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:18.515556 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:18.515558 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:18.515580 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:18.517001 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.517358 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.517381 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.517383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.517492 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.517721 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.517722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.517724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.517937 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:18.518009 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:18.518263 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 19:06:18.518358 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":30}
I20260812 19:06:18.519243 10077 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.60.keytab.
Entry for principal kudu/127.25.130.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.60.keytab.
I20260812 19:06:18.519984 10085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.519700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:39377 (local address 127.0.0.1:38676)
0812 19:06:18.519819 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:18.519916 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:39377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 19:06:18.520501 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.60:39377
--webserver_interface=127.25.130.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.60.keytab
--principal=kudu/127.25.130.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:39105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:37403,127.25.130.61:33851,127.25.130.60:39377
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:06:18.520603 10019 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.130.60:39377: connect: Connection refused (error 111)
I20260812 19:06:18.521572 10085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.521297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:39377 (local address 127.0.0.1:38678)
0812 19:06:18.521422 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:18.521498 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:39377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 19:06:18.521705 10077 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.130.60:39377: Network error: Client connection negotiation failed: client connection to 127.25.130.60:39377: connect: Connection refused (error 111)
I20260812 19:06:18.543043 9990 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 } attempt: 1
I20260812 19:06:18.543869 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.543578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:39377 (local address 127.0.0.1:38682)
0812 19:06:18.543722 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:18.543785 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:39377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 19:06:18.544623 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.544425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:39377 (local address 127.0.0.1:38694)
0812 19:06:18.544517 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:18.544561 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:39377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 19:06:18.544749 9990 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.130.60:39377: Network error: Client connection negotiation failed: client connection to 127.25.130.60:39377: connect: Connection refused (error 111)
I20260812 19:06:18.570989 10077 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 } attempt: 1
I20260812 19:06:18.571967 10085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.571609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:39377 (local address 127.0.0.1:38706)
0812 19:06:18.571781 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:18.571865 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:39377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 19:06:18.572763 10085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.572497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:39377 (local address 127.0.0.1:38708)
0812 19:06:18.572627 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:18.572688 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:39377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 19:06:18.572901 10077 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.130.60:39377: Network error: Client connection negotiation failed: client connection to 127.25.130.60:39377: connect: Connection refused (error 111)
I20260812 19:06:18.619907 9990 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 } attempt: 2
I20260812 19:06:18.620769 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.620459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:39377 (local address 127.0.0.1:38712)
0812 19:06:18.620619 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:18.620685 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:39377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260812 19:06:18.621569 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.621337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:39377 (local address 127.0.0.1:38718)
0812 19:06:18.621427 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:18.621501 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:39377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 19:06:18.621707 9990 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.130.60:39377: Network error: Client connection negotiation failed: client connection to 127.25.130.60:39377: connect: Connection refused (error 111)
W20260812 19:06:18.636325 10093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:06:18.636615 10093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:06:18.636664 10093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:06:18.640738 10093 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:06:18.640806 10093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:06:18.640825 10093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:06:18.640839 10093 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:06:18.640853 10093 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:06:18.641064 10077 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 } attempt: 2
I20260812 19:06:18.641983 10085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.641640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:39377 (local address 127.0.0.1:38734)
0812 19:06:18.641811 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:18.641888 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:39377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 19:06:18.642701 10085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.642465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:39377 (local address 127.0.0.1:38740)
0812 19:06:18.642583 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:18.642635 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:39377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 19:06:18.642815 10077 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.130.60:39377: Network error: Client connection negotiation failed: client connection to 127.25.130.60:39377: connect: Connection refused (error 111)
I20260812 19:06:18.646714 10093 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:39105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:37403,127.25.130.61:33851,127.25.130.60:39377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.60.keytab
--principal=kudu/127.25.130.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.60:39377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.25.130.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:06:18.647818 10093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:06:18.648829 10093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:18.657377 10102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:18.657408 10099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:18.657430 10100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:18.657616 10093 server_base.cc:1034] running on GCE node
I20260812 19:06:18.658329 10093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:06:18.659446 10093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:06:18.660646 10093 hybrid_clock.cc:648] HybridClock initialized: now 1786561578660621 us; error 36 us; skew 500 ppm
Aug 12 19:06:18 dist-test-slave-49h3 krb5kdc[9896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561578, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:18.663483 10093 init.cc:377] Logged in from keytab as kudu/127.25.130.60@KRBTEST.COM (short username kudu)
I20260812 19:06:18.664469 10093 webserver.cc:460] Webserver started at http://127.25.130.60:41567/ using document root <none> and password file <none>
I20260812 19:06:18.665038 10093 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:18.665110 10093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:18.665334 10093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:06:18.667645 10093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-2/data/instance:
uuid: "7d93c3fd1b764d7f8ee8b654cbc55000"
format_stamp: "Formatted at 2026-08-12 19:06:18 on dist-test-slave-49h3"
server_key: "a58265f1ab33b90c329062424ed3d580"
server_key_iv: "bdb8cad158328445c6db07fe6752ced2"
server_key_version: "encryptionkey@0"
I20260812 19:06:18.668239 10093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-2/wal/instance:
uuid: "7d93c3fd1b764d7f8ee8b654cbc55000"
format_stamp: "Formatted at 2026-08-12 19:06:18 on dist-test-slave-49h3"
server_key: "a58265f1ab33b90c329062424ed3d580"
server_key_iv: "bdb8cad158328445c6db07fe6752ced2"
server_key_version: "encryptionkey@0"
I20260812 19:06:18.672106 10093 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:06:18.674592 10109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:18.675908 10093 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 19:06:18.676031 10093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-2/wal
uuid: "7d93c3fd1b764d7f8ee8b654cbc55000"
format_stamp: "Formatted at 2026-08-12 19:06:18 on dist-test-slave-49h3"
server_key: "a58265f1ab33b90c329062424ed3d580"
server_key_iv: "bdb8cad158328445c6db07fe6752ced2"
server_key_version: "encryptionkey@0"
I20260812 19:06:18.676134 10093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:18.698611 10093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:18.701576 10093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:06:18.701766 10093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:18.708902 10093 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.60:39377
I20260812 19:06:18.708917 10170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.60:39377 every 8 connection(s)
I20260812 19:06:18.709900 10093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-2/data/info.pb
I20260812 19:06:18.712718 10172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.716578 10172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37403 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:06:18.718214 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 10093
I20260812 19:06:18.718312 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561415988404-26120-0/minicluster-data/master-2/wal/instance
I20260812 19:06:18.718598 26120 external_mini_cluster.cc:1428] Setting key 8fa84fdb8119932618ba486864f9ffaa
I20260812 19:06:18.720840 9990 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 } attempt: 3
Aug 12 19:06:18 dist-test-slave-49h3 krb5kdc[9896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561578, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for kudu/127.25.130.60@KRBTEST.COM
Aug 12 19:06:18 dist-test-slave-49h3 krb5kdc[9896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561578, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.60@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
Aug 12 19:06:18 dist-test-slave-49h3 krb5kdc[9896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.60@KRBTEST.COM
I20260812 19:06:18.731359 9993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.718555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47170 (local address 127.25.130.62:37403)
0812 19:06:18.718717 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:18.718722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:18.719809 (+ 1087us) server_negotiation.cc:407] Connection header received
0812 19:06:18.720799 (+ 990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:18.720803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:18.720862 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:18.720939 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:18.722193 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.722882 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.724352 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.724666 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.724772 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:18.727440 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:18.727444 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:18.727447 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:18.727474 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:18.729000 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.729833 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.729836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.729838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.729920 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.730246 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.730249 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.730251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.730523 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:18.730610 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:18.731021 (+ 411us) server_negotiation.cc:299] Negotiation successful
0812 19:06:18.731178 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":55}
I20260812 19:06:18.731779 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.721247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:39377 (local address 127.0.0.1:38760)
0812 19:06:18.721816 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:18.721831 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:18.721910 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:18.722308 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:18.722311 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:18.722500 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:06:18.722606 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:18.722612 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.723537 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.723541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:18.724081 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:18.724092 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.724819 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.724823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:18.724901 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:18.725396 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:06:18.725414 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:06:18.726748 (+ 1334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:06:18.729200 (+ 2452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:18.729202 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:18.729205 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:18.729399 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:18.729974 (+ 575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:18.729978 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:18.729980 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:18.730093 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:18.730829 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:06:18.730831 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:06:18.730922 (+ 91us) client_negotiation.cc:769] Sending connection context
0812 19:06:18.731006 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 19:06:18.731630 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":461}
I20260812 19:06:18.732378 10178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.721577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38760 (local address 127.25.130.60:39377)
0812 19:06:18.722077 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:18.722095 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:18.722109 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:06:18.722140 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:18.722143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:18.722180 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:18.722278 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:18.722728 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.723440 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.724189 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.724698 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.725102 (+ 404us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:18.726896 (+ 1794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:18.726904 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:18.726918 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:18.726945 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:18.729120 (+ 2175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.729525 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.729531 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.729533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.729863 (+ 330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.730206 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.730209 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.730211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.730728 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:18.730856 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:18.731187 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 19:06:18.731342 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":434,"spinlock_wait_cycles":376704,"thread_start_us":135,"threads_started":1}
I20260812 19:06:18.732378 10174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.719022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:37403 (local address 127.0.0.1:47170)
0812 19:06:18.719638 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:18.719702 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:18.720587 (+ 885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:18.721189 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:18.721194 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:18.721716 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:06:18.722062 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:18.722075 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.722982 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.722986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:18.724210 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:18.724221 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.724822 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.724826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:18.724976 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:18.725771 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:06:18.725793 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:06:18.727280 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:06:18.729136 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:18.729147 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:18.729158 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:18.729731 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:18.730043 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:18.730046 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:18.730047 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:18.730166 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:18.730624 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:06:18.730631 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:06:18.730771 (+ 140us) client_negotiation.cc:769] Sending connection context
0812 19:06:18.730945 (+ 174us) client_negotiation.cc:240] Negotiation successful
0812 19:06:18.731292 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":181,"threads_started":1}
I20260812 19:06:18.732659 10177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.719981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38754 (local address 127.25.130.60:39377)
0812 19:06:18.720253 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:18.720262 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:18.720295 (+ 33us) server_negotiation.cc:407] Connection header received
0812 19:06:18.720591 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:18.720607 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:18.720794 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:18.720961 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:18.721950 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.723245 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.724359 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.724686 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.725163 (+ 477us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:18.728177 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:18.728181 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:18.728184 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:18.728210 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:18.730284 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.730944 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.730947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.730949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.731048 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.731369 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.731372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.731373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.731572 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:18.732332 (+ 760us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:18.732399 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 19:06:18.732496 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":185,"thread_start_us":88,"threads_started":1}
I20260812 19:06:18.733955 10077 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 } attempt: 3
I20260812 19:06:18.735056 10172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.61" port: 33851 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:06:18.737601 9990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb494dfef55d46d384f4359f46e8a813: Bootstrap starting.
Aug 12 19:06:18 dist-test-slave-49h3 krb5kdc[9896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561578, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.61@KRBTEST.COM for kudu/127.25.130.60@KRBTEST.COM
I20260812 19:06:18.740300 9990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb494dfef55d46d384f4359f46e8a813: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:18.741291 9990 log.cc:824] T 00000000000000000000000000000000 P fb494dfef55d46d384f4359f46e8a813: Log is configured to *not* fsync() on all Append() calls
I20260812 19:06:18.742775 10085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.734465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:39377 (local address 127.0.0.1:38766)
0812 19:06:18.734622 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:18.734640 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:18.734731 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:18.735035 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:18.735037 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:18.735239 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:06:18.735354 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:18.735362 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.736312 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.736315 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:18.736838 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:18.736846 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.737396 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.737399 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:18.737490 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:18.737985 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:06:18.738000 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:06:18.739429 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:06:18.741229 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:18.741232 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:18.741234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:18.741482 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:18.741775 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:18.741778 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:18.741780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:18.741875 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:18.742270 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:06:18.742273 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:06:18.742368 (+ 95us) client_negotiation.cc:769] Sending connection context
0812 19:06:18.742470 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 19:06:18.742605 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 19:06:18.742828 10178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.734729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38766 (local address 127.25.130.60:39377)
0812 19:06:18.734830 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:18.734834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:18.734846 (+ 12us) server_negotiation.cc:407] Connection header received
0812 19:06:18.734879 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:18.734881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:18.734925 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:18.734996 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:18.735495 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.736213 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.736953 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.737295 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.737402 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:18.739575 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:18.739578 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:18.739580 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:18.739603 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:18.741108 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.741589 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.741593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.741594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.741690 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.741979 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.741982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.741984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.742175 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:18.742262 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:18.742541 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 19:06:18.742654 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 19:06:18.745994 9990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb494dfef55d46d384f4359f46e8a813: No bootstrap required, opened a new log
Aug 12 19:06:18 dist-test-slave-49h3 krb5kdc[9896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561578, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.60@KRBTEST.COM for kudu/127.25.130.61@KRBTEST.COM
Aug 12 19:06:18 dist-test-slave-49h3 krb5kdc[9896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:06:18.748317 10077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45: Bootstrap starting.
I20260812 19:06:18.750300 10083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.741583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60396 (local address 127.25.130.61:33851)
0812 19:06:18.741718 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:18.741721 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:18.741732 (+ 11us) server_negotiation.cc:407] Connection header received
0812 19:06:18.741765 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:18.741767 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:18.741810 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:18.741892 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:18.742353 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.743095 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.744095 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.744394 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.744475 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:18.746823 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:18.746827 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:18.746829 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:18.746856 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:18.748403 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.748879 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.748883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.748884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.748973 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.749285 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.749288 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.749289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.749552 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:18.749623 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:18.750000 (+ 377us) server_negotiation.cc:299] Negotiation successful
0812 19:06:18.750133 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 19:06:18.750489 9990 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb494dfef55d46d384f4359f46e8a813 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb494dfef55d46d384f4359f46e8a813" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37403 } } peers { permanent_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 33851 } } peers { permanent_uuid: "7d93c3fd1b764d7f8ee8b654cbc55000" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 } }
I20260812 19:06:18.750780 10174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.735518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:33851 (local address 127.0.0.1:60396)
0812 19:06:18.735743 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:18.735756 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:18.735820 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:18.741917 (+ 6097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:18.741921 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:18.742087 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:06:18.742210 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:18.742216 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.743201 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.743204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:18.743943 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:18.743954 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.744623 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.744627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:18.744734 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:18.745197 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:06:18.745210 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:06:18.746668 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:06:18.748507 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:18.748510 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:18.748513 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:18.748763 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:18.749089 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:18.749092 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:18.749094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:18.749184 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:18.749697 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:06:18.749699 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:06:18.749804 (+ 105us) client_negotiation.cc:769] Sending connection context
0812 19:06:18.750456 (+ 652us) client_negotiation.cc:240] Negotiation successful
0812 19:06:18.750626 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160}
I20260812 19:06:18.750833 9990 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb494dfef55d46d384f4359f46e8a813 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:18.750895 9990 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb494dfef55d46d384f4359f46e8a813 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb494dfef55d46d384f4359f46e8a813, State: Initialized, Role: FOLLOWER
I20260812 19:06:18.751291 9993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.736059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47172 (local address 127.25.130.62:37403)
0812 19:06:18.741749 (+ 5690us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:18.741754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:18.741768 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:06:18.741805 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:18.741807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:18.741848 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:18.741919 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:18.742804 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.743573 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.744588 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.744871 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.744949 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:18.747668 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:18.747671 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:18.747674 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:18.747703 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:18.749252 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.749796 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.749800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.749802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.749903 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.750236 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.750240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.750242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.750461 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:18.750610 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:18.751021 (+ 411us) server_negotiation.cc:299] Negotiation successful
0812 19:06:18.751152 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5603}
I20260812 19:06:18.751819 10172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:06:18.751472 9990 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb494dfef55d46d384f4359f46e8a813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb494dfef55d46d384f4359f46e8a813" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37403 } } peers { permanent_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 33851 } } peers { permanent_uuid: "7d93c3fd1b764d7f8ee8b654cbc55000" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 } }
I20260812 19:06:18.753086 10077 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:18.753337 9990 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb494dfef55d46d384f4359f46e8a813 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:06:18.754164 10077 log.cc:824] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45: Log is configured to *not* fsync() on all Append() calls
I20260812 19:06:18.755810 10180 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb494dfef55d46d384f4359f46e8a813 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb494dfef55d46d384f4359f46e8a813" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37403 } } peers { permanent_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 33851 } } peers { permanent_uuid: "7d93c3fd1b764d7f8ee8b654cbc55000" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 } } }
I20260812 19:06:18.755963 10180 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb494dfef55d46d384f4359f46e8a813 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:06:18.757043 10077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45: No bootstrap required, opened a new log
Aug 12 19:06:18 dist-test-slave-49h3 krb5kdc[9896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561578, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.60@KRBTEST.COM for kudu/127.25.130.60@KRBTEST.COM
Aug 12 19:06:18 dist-test-slave-49h3 krb5kdc[9896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.61@KRBTEST.COM
W20260812 19:06:18.759917 10194 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fb494dfef55d46d384f4359f46e8a813: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:06:18.760016 10194 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:06:18.761137 10174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.752294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:39377 (local address 127.0.0.1:38782)
0812 19:06:18.752614 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:18.752627 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:18.752707 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:18.753049 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:18.753052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:18.753222 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:06:18.753327 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:18.753334 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.754338 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.754341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:18.755067 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:18.755079 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.755690 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.755693 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:18.755761 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:18.756233 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:06:18.756248 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:06:18.757815 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:06:18.759591 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:18.759594 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:18.759596 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:18.759832 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:18.760133 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:18.760136 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:18.760138 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:18.760224 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:18.760646 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:06:18.760649 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:06:18.760735 (+ 86us) client_negotiation.cc:769] Sending connection context
0812 19:06:18.760824 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 19:06:18.760967 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253}
I20260812 19:06:18.761613 10178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.752501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38782 (local address 127.25.130.60:39377)
0812 19:06:18.752823 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:18.752827 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:18.752840 (+ 13us) server_negotiation.cc:407] Connection header received
0812 19:06:18.752877 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:18.752879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:18.752923 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:18.753006 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:18.753500 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.754221 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.755193 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.755479 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.755583 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:18.757951 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:18.757954 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:18.757957 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:18.757978 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:18.759479 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.759930 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.759933 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.759935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.760027 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.760310 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.760312 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.760314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.760528 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:18.760615 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:18.761339 (+ 724us) server_negotiation.cc:299] Negotiation successful
0812 19:06:18.761476 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252}
I20260812 19:06:18.763862 10083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.752625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60406 (local address 127.25.130.61:33851)
0812 19:06:18.752731 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:18.752735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:18.753250 (+ 515us) server_negotiation.cc:407] Connection header received
0812 19:06:18.753426 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:18.753429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:18.753499 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:18.753549 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:18.754400 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.755129 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.756156 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.756478 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.756595 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:18.759588 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:18.759592 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:18.759595 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:18.759624 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:18.761176 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.762559 (+ 1383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.762563 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.762565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.762668 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.762955 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.762958 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.762960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.763179 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:18.763380 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:18.763581 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 19:06:18.763709 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 19:06:18.761588 9912 master_runner.cc:186] Error getting master registration for 127.25.130.61:33851: OK, instance_id { permanent_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" instance_seqno: 1786561578484533 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 19:06:18.764741 9912 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 19:06:18.768105 10172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d93c3fd1b764d7f8ee8b654cbc55000: Bootstrap starting.
I20260812 19:06:18.769321 10077 raft_consensus.cc:348] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb494dfef55d46d384f4359f46e8a813" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37403 } } peers { permanent_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 33851 } } peers { permanent_uuid: "7d93c3fd1b764d7f8ee8b654cbc55000" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 } }
I20260812 19:06:18.769639 10077 raft_consensus.cc:374] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:18.769706 10077 raft_consensus.cc:729] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c76b7dd5bfaa45d499a4fa0219a11e45, State: Initialized, Role: FOLLOWER
I20260812 19:06:18.770221 10077 consensus_queue.cc:260] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb494dfef55d46d384f4359f46e8a813" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37403 } } peers { permanent_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 33851 } } peers { permanent_uuid: "7d93c3fd1b764d7f8ee8b654cbc55000" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 } }
I20260812 19:06:18.770826 10197 sys_catalog.cc:455] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb494dfef55d46d384f4359f46e8a813" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37403 } } peers { permanent_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 33851 } } peers { permanent_uuid: "7d93c3fd1b764d7f8ee8b654cbc55000" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 } } }
I20260812 19:06:18.771036 10197 sys_catalog.cc:458] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:06:18.771256 10172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d93c3fd1b764d7f8ee8b654cbc55000: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:18.771739 10077 sys_catalog.cc:564] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:06:18.772207 10172 log.cc:824] T 00000000000000000000000000000000 P 7d93c3fd1b764d7f8ee8b654cbc55000: Log is configured to *not* fsync() on all Append() calls
I20260812 19:06:18.774852 26120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260812 19:06:18.777031 10213 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:06:18.777180 10213 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:06:18.778467 10172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d93c3fd1b764d7f8ee8b654cbc55000: No bootstrap required, opened a new log
I20260812 19:06:18.779198 9998 master_runner.cc:186] Error getting master registration for 127.25.130.60:39377: OK, instance_id { permanent_uuid: "7d93c3fd1b764d7f8ee8b654cbc55000" instance_seqno: 1786561578706887 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 19:06:18.783592 9998 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 19:06:18.784996 10172 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d93c3fd1b764d7f8ee8b654cbc55000 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb494dfef55d46d384f4359f46e8a813" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37403 } } peers { permanent_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 33851 } } peers { permanent_uuid: "7d93c3fd1b764d7f8ee8b654cbc55000" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 } }
I20260812 19:06:18.785300 10172 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d93c3fd1b764d7f8ee8b654cbc55000 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:18.785369 10172 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d93c3fd1b764d7f8ee8b654cbc55000 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d93c3fd1b764d7f8ee8b654cbc55000, State: Initialized, Role: FOLLOWER
I20260812 19:06:18.785975 10172 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d93c3fd1b764d7f8ee8b654cbc55000 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb494dfef55d46d384f4359f46e8a813" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37403 } } peers { permanent_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 33851 } } peers { permanent_uuid: "7d93c3fd1b764d7f8ee8b654cbc55000" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 } }
I20260812 19:06:18.787654 10172 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d93c3fd1b764d7f8ee8b654cbc55000 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:06:18.790843 10222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d93c3fd1b764d7f8ee8b654cbc55000 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb494dfef55d46d384f4359f46e8a813" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37403 } } peers { permanent_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 33851 } } peers { permanent_uuid: "7d93c3fd1b764d7f8ee8b654cbc55000" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 } } }
I20260812 19:06:18.790994 10222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d93c3fd1b764d7f8ee8b654cbc55000 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:06:18.792578 10083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.781914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60422 (local address 127.25.130.61:33851)
0812 19:06:18.782052 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:18.782057 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:18.782485 (+ 428us) server_negotiation.cc:407] Connection header received
0812 19:06:18.782711 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:18.782714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:18.782764 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:18.782813 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:18.783998 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.784794 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.786309 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.786676 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.786753 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:18.788690 (+ 1937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:18.788694 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:18.788698 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:18.788728 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:18.790549 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.791150 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.791154 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.791156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.791256 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.791583 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.791587 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.791588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.791814 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:18.791911 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:18.792268 (+ 357us) server_negotiation.cc:299] Negotiation successful
0812 19:06:18.792396 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 19:06:18.796806 10093 master_runner.cc:263] No leader master found from master 7d93c3fd1b764d7f8ee8b654cbc55000
I20260812 19:06:18.796881 10093 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 19:06:18.798095 10234 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7d93c3fd1b764d7f8ee8b654cbc55000: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:06:18.798187 10234 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:06:18.807299 9993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.783319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47182 (local address 127.25.130.62:37403)
0812 19:06:18.783445 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:18.783450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:18.790373 (+ 6923us) server_negotiation.cc:407] Connection header received
0812 19:06:18.790432 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:18.790435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:18.790490 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:18.790559 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:18.792024 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.792692 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.793856 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.794222 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.794302 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:18.801714 (+ 7412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:18.801718 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:18.801721 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:18.801759 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:18.803974 (+ 2215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.804831 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.804835 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.804837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.804940 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.806224 (+ 1284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.806229 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.806231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.806478 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:18.806617 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:18.806977 (+ 360us) server_negotiation.cc:299] Negotiation successful
0812 19:06:18.807121 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 19:06:18.812844 10178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:18.782576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38784 (local address 127.25.130.60:39377)
0812 19:06:18.782702 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:18.782706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:18.787800 (+ 5094us) server_negotiation.cc:407] Connection header received
0812 19:06:18.787983 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:18.787986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:18.788037 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:18.788091 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:18.789275 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.790278 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.802143 (+ 11865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:18.802512 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:18.802602 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:18.809134 (+ 6532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:18.809139 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:18.809142 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:18.809176 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:18.810885 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.811396 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.811401 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.811403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.811497 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:18.811801 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:18.811805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:18.811807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:18.812053 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:18.812167 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:18.812525 (+ 358us) server_negotiation.cc:299] Negotiation successful
0812 19:06:18.812669 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260812 19:06:18.953253 10117 tablet.cc:2369] T 00000000000000000000000000000000 P 7d93c3fd1b764d7f8ee8b654cbc55000: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 19:06:18.981253 10025 tablet.cc:2369] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 19:06:18.993235 9937 tablet.cc:2369] T 00000000000000000000000000000000 P fb494dfef55d46d384f4359f46e8a813: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:06:19.031540 10197 raft_consensus.cc:482] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:19.031744 10197 raft_consensus.cc:504] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb494dfef55d46d384f4359f46e8a813" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37403 } } peers { permanent_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 33851 } } peers { permanent_uuid: "7d93c3fd1b764d7f8ee8b654cbc55000" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 } }
I20260812 19:06:19.032663 10197 leader_election.cc:288] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb494dfef55d46d384f4359f46e8a813 (127.25.130.62:37403), 7d93c3fd1b764d7f8ee8b654cbc55000 (127.25.130.60:39377)
I20260812 19:06:19.033229 10139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d93c3fd1b764d7f8ee8b654cbc55000" is_pre_election: true
I20260812 19:06:19.033218 9963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb494dfef55d46d384f4359f46e8a813" is_pre_election: true
I20260812 19:06:19.033571 9963 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fb494dfef55d46d384f4359f46e8a813 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c76b7dd5bfaa45d499a4fa0219a11e45 in term 0.
I20260812 19:06:19.033576 10139 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7d93c3fd1b764d7f8ee8b654cbc55000 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c76b7dd5bfaa45d499a4fa0219a11e45 in term 0.
I20260812 19:06:19.034035 10019 leader_election.cc:302] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7d93c3fd1b764d7f8ee8b654cbc55000, c76b7dd5bfaa45d499a4fa0219a11e45, fb494dfef55d46d384f4359f46e8a813; no voters:
I20260812 19:06:19.034340 10197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:19.034430 10197 raft_consensus.cc:482] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:19.034459 10197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:19.035530 10197 raft_consensus.cc:504] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb494dfef55d46d384f4359f46e8a813" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37403 } } peers { permanent_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 33851 } } peers { permanent_uuid: "7d93c3fd1b764d7f8ee8b654cbc55000" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 } }
I20260812 19:06:19.035868 10197 leader_election.cc:288] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45 [CANDIDATE]: Term 1 election: Requested vote from peers fb494dfef55d46d384f4359f46e8a813 (127.25.130.62:37403), 7d93c3fd1b764d7f8ee8b654cbc55000 (127.25.130.60:39377)
I20260812 19:06:19.036270 9963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb494dfef55d46d384f4359f46e8a813"
I20260812 19:06:19.036273 10139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d93c3fd1b764d7f8ee8b654cbc55000"
I20260812 19:06:19.036398 10139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d93c3fd1b764d7f8ee8b654cbc55000 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:19.036391 9963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb494dfef55d46d384f4359f46e8a813 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:19.037492 9963 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fb494dfef55d46d384f4359f46e8a813 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c76b7dd5bfaa45d499a4fa0219a11e45 in term 1.
I20260812 19:06:19.037493 10139 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7d93c3fd1b764d7f8ee8b654cbc55000 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c76b7dd5bfaa45d499a4fa0219a11e45 in term 1.
I20260812 19:06:19.037801 10022 leader_election.cc:302] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c76b7dd5bfaa45d499a4fa0219a11e45, fb494dfef55d46d384f4359f46e8a813; no voters:
I20260812 19:06:19.038017 10197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:19.038223 10197 raft_consensus.cc:686] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45 [term 1 LEADER]: Becoming Leader. State: Replica: c76b7dd5bfaa45d499a4fa0219a11e45, State: Running, Role: LEADER
I20260812 19:06:19.038549 10197 consensus_queue.cc:237] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb494dfef55d46d384f4359f46e8a813" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37403 } } peers { permanent_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 33851 } } peers { permanent_uuid: "7d93c3fd1b764d7f8ee8b654cbc55000" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 } }
I20260812 19:06:19.040484 10236 sys_catalog.cc:455] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c76b7dd5bfaa45d499a4fa0219a11e45. Latest consensus state: current_term: 1 leader_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb494dfef55d46d384f4359f46e8a813" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37403 } } peers { permanent_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 33851 } } peers { permanent_uuid: "7d93c3fd1b764d7f8ee8b654cbc55000" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 } } }
I20260812 19:06:19.040634 10236 sys_catalog.cc:458] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45 [sys.catalog]: This master's current role is: LEADER
I20260812 19:06:19.041004 10238 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:06:19.041910 10238 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:06:19.046484 10139 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7d93c3fd1b764d7f8ee8b654cbc55000 [term 1 FOLLOWER]: Refusing update from remote peer c76b7dd5bfaa45d499a4fa0219a11e45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:19.046489 9963 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fb494dfef55d46d384f4359f46e8a813 [term 1 FOLLOWER]: Refusing update from remote peer c76b7dd5bfaa45d499a4fa0219a11e45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:19.047129 10236 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb494dfef55d46d384f4359f46e8a813" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:19.047295 10197 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d93c3fd1b764d7f8ee8b654cbc55000" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:19.051468 10180 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb494dfef55d46d384f4359f46e8a813 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c76b7dd5bfaa45d499a4fa0219a11e45. Latest consensus state: current_term: 1 leader_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb494dfef55d46d384f4359f46e8a813" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37403 } } peers { permanent_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 33851 } } peers { permanent_uuid: "7d93c3fd1b764d7f8ee8b654cbc55000" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 } } }
I20260812 19:06:19.051640 10180 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb494dfef55d46d384f4359f46e8a813 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:06:19.053005 10222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d93c3fd1b764d7f8ee8b654cbc55000 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c76b7dd5bfaa45d499a4fa0219a11e45. Latest consensus state: current_term: 1 leader_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb494dfef55d46d384f4359f46e8a813" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37403 } } peers { permanent_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 33851 } } peers { permanent_uuid: "7d93c3fd1b764d7f8ee8b654cbc55000" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 } } }
I20260812 19:06:19.053146 10222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d93c3fd1b764d7f8ee8b654cbc55000 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:06:19.053679 10236 sys_catalog.cc:455] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb494dfef55d46d384f4359f46e8a813" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37403 } } peers { permanent_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 33851 } } peers { permanent_uuid: "7d93c3fd1b764d7f8ee8b654cbc55000" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 } } }
I20260812 19:06:19.053802 10236 sys_catalog.cc:458] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45 [sys.catalog]: This master's current role is: LEADER
I20260812 19:06:19.054023 10240 mvcc.cc:205] Tried to move back new op lower bound from 7317756227767599104 to 7317756227744628736. Current Snapshot: MvccSnapshot[applied={T|T < 7317756227767599104}]
I20260812 19:06:19.054657 10180 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb494dfef55d46d384f4359f46e8a813 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb494dfef55d46d384f4359f46e8a813" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37403 } } peers { permanent_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 33851 } } peers { permanent_uuid: "7d93c3fd1b764d7f8ee8b654cbc55000" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 } } }
I20260812 19:06:19.054781 10180 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb494dfef55d46d384f4359f46e8a813 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:06:19.055147 10238 catalog_manager.cc:1347] Generated new cluster ID: 3220513a045d4894849346593a1ed5cc
I20260812 19:06:19.055208 10238 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:06:19.056648 10236 sys_catalog.cc:455] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb494dfef55d46d384f4359f46e8a813" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37403 } } peers { permanent_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 33851 } } peers { permanent_uuid: "7d93c3fd1b764d7f8ee8b654cbc55000" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 } } }
I20260812 19:06:19.056761 10236 sys_catalog.cc:458] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45 [sys.catalog]: This master's current role is: LEADER
I20260812 19:06:19.057582 10222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d93c3fd1b764d7f8ee8b654cbc55000 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb494dfef55d46d384f4359f46e8a813" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 37403 } } peers { permanent_uuid: "c76b7dd5bfaa45d499a4fa0219a11e45" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 33851 } } peers { permanent_uuid: "7d93c3fd1b764d7f8ee8b654cbc55000" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 39377 } } }
I20260812 19:06:19.057708 10222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d93c3fd1b764d7f8ee8b654cbc55000 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:06:19.067528 10238 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:06:19.068711 10238 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:06:19.080926 10238 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c76b7dd5bfaa45d499a4fa0219a11e45: Generated new TSK 0
I20260812 19:06:19.081941 10238 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:06:19.761433 10194 catalog_manager.cc:1259] Loaded cluster ID: 3220513a045d4894849346593a1ed5cc
I20260812 19:06:19.761562 10194 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fb494dfef55d46d384f4359f46e8a813: loading cluster ID for follower catalog manager: success
I20260812 19:06:19.763345 10194 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fb494dfef55d46d384f4359f46e8a813: acquiring CA information for follower catalog manager: success
I20260812 19:06:19.764712 10194 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fb494dfef55d46d384f4359f46e8a813: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 19:06:19.799521 10234 catalog_manager.cc:1259] Loaded cluster ID: 3220513a045d4894849346593a1ed5cc
I20260812 19:06:19.799623 10234 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7d93c3fd1b764d7f8ee8b654cbc55000: loading cluster ID for follower catalog manager: success
I20260812 19:06:19.801265 10234 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7d93c3fd1b764d7f8ee8b654cbc55000: acquiring CA information for follower catalog manager: success
I20260812 19:06:19.802456 10234 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7d93c3fd1b764d7f8ee8b654cbc55000: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 19:06:20.153275 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 9912
I20260812 19:06:20.161816 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 9998
I20260812 19:06:20.170995 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 10093
2026-08-12T19:06:20Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3309 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:06:20 dist-test-slave-49h3 krb5kdc[10258](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:06:20 dist-test-slave-49h3 krb5kdc[10258](info): set up 2 sockets
Aug 12 19:06:20 dist-test-slave-49h3 krb5kdc[10258](info): commencing operation
krb5kdc: starting...
W20260812 19:06:21.231556 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:06:21 dist-test-slave-49h3 krb5kdc[10258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:06:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:06:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:06:21.356920 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:39063
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:39369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:39063
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:06:21.471947 10273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:06:21.472276 10273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:06:21.472337 10273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:06:21.476581 10273 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:06:21.476675 10273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:06:21.476703 10273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:06:21.476724 10273 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:06:21.476747 10273 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:06:21.482728 10273 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:39369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:39063
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:39063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:06:21.483907 10273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:06:21.484954 10273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:21.493495 10282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:21.493506 10280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:21.493587 10273 server_base.cc:1034] running on GCE node
W20260812 19:06:21.493495 10279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:21.494444 10273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:06:21.495558 10273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:06:21.496722 10273 hybrid_clock.cc:648] HybridClock initialized: now 1786561581496709 us; error 27 us; skew 500 ppm
Aug 12 19:06:21 dist-test-slave-49h3 krb5kdc[10258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561581, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:21.499579 10273 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:06:21.500516 10273 webserver.cc:460] Webserver started at http://127.25.130.62:34075/ using document root <none> and password file <none>
I20260812 19:06:21.501049 10273 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:21.501109 10273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:21.501297 10273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:06:21.503561 10273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "8ad48bbca43f42c1a4c80cd7de976bf8"
format_stamp: "Formatted at 2026-08-12 19:06:21 on dist-test-slave-49h3"
server_key: "d15993c2400552f478a301e6e956f8f1"
server_key_iv: "cc4588207a9141106be8ae14755431a5"
server_key_version: "encryptionkey@0"
I20260812 19:06:21.504108 10273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "8ad48bbca43f42c1a4c80cd7de976bf8"
format_stamp: "Formatted at 2026-08-12 19:06:21 on dist-test-slave-49h3"
server_key: "d15993c2400552f478a301e6e956f8f1"
server_key_iv: "cc4588207a9141106be8ae14755431a5"
server_key_version: "encryptionkey@0"
I20260812 19:06:21.508021 10273 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:06:21.510558 10290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:21.511852 10273 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:06:21.511997 10273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "8ad48bbca43f42c1a4c80cd7de976bf8"
format_stamp: "Formatted at 2026-08-12 19:06:21 on dist-test-slave-49h3"
server_key: "d15993c2400552f478a301e6e956f8f1"
server_key_iv: "cc4588207a9141106be8ae14755431a5"
server_key_version: "encryptionkey@0"
I20260812 19:06:21.512115 10273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:21.529526 10273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:21.532917 10273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:06:21.533121 10273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:21.540668 10273 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:39063
I20260812 19:06:21.540697 10357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:39063 every 8 connection(s)
I20260812 19:06:21.541761 10273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:06:21.544186 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 10273
I20260812 19:06:21.544317 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:06:21.544627 26120 external_mini_cluster.cc:1428] Setting key fb73b9e86a2f78de52892bccc37cd2db
I20260812 19:06:21.544939 10358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:21.552117 10358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ad48bbca43f42c1a4c80cd7de976bf8: Bootstrap starting.
Aug 12 19:06:21 dist-test-slave-49h3 krb5kdc[10258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:06:21.554787 10358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ad48bbca43f42c1a4c80cd7de976bf8: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:21.555599 10358 log.cc:824] T 00000000000000000000000000000000 P 8ad48bbca43f42c1a4c80cd7de976bf8: Log is configured to *not* fsync() on all Append() calls
I20260812 19:06:21.557960 10358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ad48bbca43f42c1a4c80cd7de976bf8: No bootstrap required, opened a new log
I20260812 19:06:21.559243 10363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:21.546062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54692 (local address 127.25.130.62:39063)
0812 19:06:21.546493 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:21.546509 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:21.546550 (+ 41us) server_negotiation.cc:407] Connection header received
0812 19:06:21.547226 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:21.547244 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:21.547580 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:21.547869 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:21.548791 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:21.549864 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:21.551018 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:21.551503 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:21.551619 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:21.554412 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:21.554418 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:21.554432 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:21.554461 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:21.556161 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:21.556615 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:21.556621 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:21.556627 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:21.556774 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:21.556996 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:21.556999 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:21.557001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:21.557567 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:21.557755 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:21.557960 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 19:06:21.558287 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":127,"threads_started":1}
I20260812 19:06:21.561902 10358 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ad48bbca43f42c1a4c80cd7de976bf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad48bbca43f42c1a4c80cd7de976bf8" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39063 } }
I20260812 19:06:21.562155 10358 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ad48bbca43f42c1a4c80cd7de976bf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:21.562222 10358 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ad48bbca43f42c1a4c80cd7de976bf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ad48bbca43f42c1a4c80cd7de976bf8, State: Initialized, Role: FOLLOWER
I20260812 19:06:21.562742 26120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:06:21.562686 10358 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ad48bbca43f42c1a4c80cd7de976bf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad48bbca43f42c1a4c80cd7de976bf8" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39063 } }
I20260812 19:06:21.562840 10358 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ad48bbca43f42c1a4c80cd7de976bf8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:06:21.562908 10358 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ad48bbca43f42c1a4c80cd7de976bf8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:06:21.563006 10358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ad48bbca43f42c1a4c80cd7de976bf8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:21.564035 10358 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ad48bbca43f42c1a4c80cd7de976bf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad48bbca43f42c1a4c80cd7de976bf8" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39063 } }
I20260812 19:06:21.564388 10358 leader_election.cc:302] T 00000000000000000000000000000000 P 8ad48bbca43f42c1a4c80cd7de976bf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ad48bbca43f42c1a4c80cd7de976bf8; no voters:
I20260812 19:06:21.564702 10358 leader_election.cc:288] T 00000000000000000000000000000000 P 8ad48bbca43f42c1a4c80cd7de976bf8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:06:21.564845 10365 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ad48bbca43f42c1a4c80cd7de976bf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:21.565060 10365 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ad48bbca43f42c1a4c80cd7de976bf8 [term 1 LEADER]: Becoming Leader. State: Replica: 8ad48bbca43f42c1a4c80cd7de976bf8, State: Running, Role: LEADER
I20260812 19:06:21.565388 10365 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ad48bbca43f42c1a4c80cd7de976bf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad48bbca43f42c1a4c80cd7de976bf8" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39063 } }
I20260812 19:06:21.565932 10358 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ad48bbca43f42c1a4c80cd7de976bf8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:06:21.567020 10366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ad48bbca43f42c1a4c80cd7de976bf8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ad48bbca43f42c1a4c80cd7de976bf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad48bbca43f42c1a4c80cd7de976bf8" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39063 } } }
I20260812 19:06:21.567168 10366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ad48bbca43f42c1a4c80cd7de976bf8 [sys.catalog]: This master's current role is: LEADER
I20260812 19:06:21.567485 10367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ad48bbca43f42c1a4c80cd7de976bf8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ad48bbca43f42c1a4c80cd7de976bf8. Latest consensus state: current_term: 1 leader_uuid: "8ad48bbca43f42c1a4c80cd7de976bf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad48bbca43f42c1a4c80cd7de976bf8" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39063 } } }
I20260812 19:06:21.567572 10375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:06:21.567667 10367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ad48bbca43f42c1a4c80cd7de976bf8 [sys.catalog]: This master's current role is: LEADER
I20260812 19:06:21.570899 10375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:06:21.576859 10375 catalog_manager.cc:1347] Generated new cluster ID: 0f028c93505a4c9684df11fada7a7249
I20260812 19:06:21.576942 10375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:06:21.607722 10375 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:06:21.609052 10375 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:06:21.625885 10375 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ad48bbca43f42c1a4c80cd7de976bf8: Generated new TSK 0
I20260812 19:06:21.626904 10375 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:06:21.669953 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 10273
2026-08-12T19:06:21Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1497 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:06:21 dist-test-slave-49h3 krb5kdc[10388](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:06:21 dist-test-slave-49h3 krb5kdc[10388](info): set up 2 sockets
Aug 12 19:06:21 dist-test-slave-49h3 krb5kdc[10388](info): commencing operation
krb5kdc: starting...
W20260812 19:06:22.735124 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:06:22 dist-test-slave-49h3 krb5kdc[10388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:06:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:06:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:06:22.860330 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:43125
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:35319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:43125
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:06:22.974933 10404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:06:22.975248 10404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:06:22.975307 10404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:06:22.979462 10404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:06:22.979542 10404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:06:22.979569 10404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:06:22.979589 10404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:06:22.979610 10404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:06:22.985196 10404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:35319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:43125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:43125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:06:22.986390 10404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:06:22.987465 10404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:22.996372 10414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:22.996372 10409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:22.996400 10411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:22.996793 10404 server_base.cc:1034] running on GCE node
I20260812 19:06:22.997417 10404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:06:22.998649 10404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:06:22.999833 10404 hybrid_clock.cc:648] HybridClock initialized: now 1786561582999812 us; error 33 us; skew 500 ppm
Aug 12 19:06:23 dist-test-slave-49h3 krb5kdc[10388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561583, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:23.002671 10404 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:06:23.003623 10404 webserver.cc:460] Webserver started at http://127.25.130.62:46101/ using document root <none> and password file <none>
I20260812 19:06:23.004172 10404 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:23.004231 10404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:23.004412 10404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:06:23.006575 10404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "e6b8bf975275492ab331eb132c0d92b7"
format_stamp: "Formatted at 2026-08-12 19:06:23 on dist-test-slave-49h3"
server_key: "e871a9b3f2cd887aeb459bca1de2a578"
server_key_iv: "f64f5fb3e4733f5f5f1b25e46c08aa5a"
server_key_version: "encryptionkey@0"
I20260812 19:06:23.007128 10404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "e6b8bf975275492ab331eb132c0d92b7"
format_stamp: "Formatted at 2026-08-12 19:06:23 on dist-test-slave-49h3"
server_key: "e871a9b3f2cd887aeb459bca1de2a578"
server_key_iv: "f64f5fb3e4733f5f5f1b25e46c08aa5a"
server_key_version: "encryptionkey@0"
I20260812 19:06:23.010881 10404 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:06:23.013370 10421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:23.014698 10404 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:06:23.014812 10404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "e6b8bf975275492ab331eb132c0d92b7"
format_stamp: "Formatted at 2026-08-12 19:06:23 on dist-test-slave-49h3"
server_key: "e871a9b3f2cd887aeb459bca1de2a578"
server_key_iv: "f64f5fb3e4733f5f5f1b25e46c08aa5a"
server_key_version: "encryptionkey@0"
I20260812 19:06:23.014912 10404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:23.045853 10404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:23.049232 10404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:06:23.049407 10404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:23.056542 10404 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:43125
I20260812 19:06:23.056860 10492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:43125 every 8 connection(s)
I20260812 19:06:23.057610 10404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:06:23.058061 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 10404
I20260812 19:06:23.058172 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:06:23.058446 26120 external_mini_cluster.cc:1428] Setting key c25b8399d8e7a250c16fb1e037c88f52
I20260812 19:06:23.060770 10493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:23.067879 10493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7: Bootstrap starting.
Aug 12 19:06:23 dist-test-slave-49h3 krb5kdc[10388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:06:23.070636 10493 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:23.071497 10493 log.cc:824] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7: Log is configured to *not* fsync() on all Append() calls
I20260812 19:06:23.073588 10493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7: No bootstrap required, opened a new log
I20260812 19:06:23.073832 10496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:23.059887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43362 (local address 127.25.130.62:43125)
0812 19:06:23.060311 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:23.060331 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:23.060378 (+ 47us) server_negotiation.cc:407] Connection header received
0812 19:06:23.061255 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:23.061281 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:23.061755 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:23.062150 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:23.062858 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:23.064415 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:23.065318 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:23.066103 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:23.066258 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:23.068777 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:23.068784 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:23.068802 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:23.068837 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:23.070766 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:23.071193 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:23.071199 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:23.071205 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:23.071354 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:23.071717 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:23.071720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:23.071721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:23.072196 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:23.072386 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:23.072706 (+ 320us) server_negotiation.cc:299] Negotiation successful
0812 19:06:23.073009 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":274,"thread_start_us":117,"threads_started":1}
I20260812 19:06:23.077149 10493 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b8bf975275492ab331eb132c0d92b7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 43125 } }
I20260812 19:06:23.077476 10493 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:23.077541 10493 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b8bf975275492ab331eb132c0d92b7, State: Initialized, Role: FOLLOWER
I20260812 19:06:23.077989 10493 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b8bf975275492ab331eb132c0d92b7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 43125 } }
I20260812 19:06:23.078127 10493 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:06:23.078193 10493 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:06:23.078289 10493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:23.079212 10493 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b8bf975275492ab331eb132c0d92b7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 43125 } }
I20260812 19:06:23.079527 10493 leader_election.cc:302] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6b8bf975275492ab331eb132c0d92b7; no voters:
I20260812 19:06:23.079831 10493 leader_election.cc:288] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:06:23.079963 10498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:23.080420 10498 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [term 1 LEADER]: Becoming Leader. State: Replica: e6b8bf975275492ab331eb132c0d92b7, State: Running, Role: LEADER
I20260812 19:06:23.080747 10498 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b8bf975275492ab331eb132c0d92b7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 43125 } }
I20260812 19:06:23.081094 10493 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:06:23.082017 10502 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6b8bf975275492ab331eb132c0d92b7. Latest consensus state: current_term: 1 leader_uuid: "e6b8bf975275492ab331eb132c0d92b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b8bf975275492ab331eb132c0d92b7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 43125 } } }
I20260812 19:06:23.082154 10502 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [sys.catalog]: This master's current role is: LEADER
I20260812 19:06:23.082412 10499 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6b8bf975275492ab331eb132c0d92b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b8bf975275492ab331eb132c0d92b7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 43125 } } }
I20260812 19:06:23.082500 10499 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [sys.catalog]: This master's current role is: LEADER
I20260812 19:06:23.082566 10510 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:06:23.085860 10510 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:06:23.091995 10510 catalog_manager.cc:1347] Generated new cluster ID: dab9e87dc88d41f4b59fc9335882d6f0
I20260812 19:06:23.092106 10510 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:06:23.100274 10510 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:06:23.101312 10510 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:06:23.110553 10510 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7: Generated new TSK 0
I20260812 19:06:23.111428 10510 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
I20260812 19:06:23.152454 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:0
--local_ip_for_outbound_sockets=127.25.130.1
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:43125
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:35319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:06:23.293987 10525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:06:23.294275 10525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:06:23.294318 10525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:06:23.298389 10525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:06:23.298461 10525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:06:23.298525 10525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:06:23.304450 10525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:35319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:43125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:06:23.305621 10525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:06:23.306676 10525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:23.316655 10531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:23.316674 10533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:23.316655 10530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:23.316919 10525 server_base.cc:1034] running on GCE node
I20260812 19:06:23.317420 10525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:06:23.318194 10525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:06:23.319380 10525 hybrid_clock.cc:648] HybridClock initialized: now 1786561583319361 us; error 36 us; skew 500 ppm
Aug 12 19:06:23 dist-test-slave-49h3 krb5kdc[10388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561583, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:23.326772 10525 init.cc:377] Logged in from keytab as kudu/127.25.130.1@KRBTEST.COM (short username kudu)
I20260812 19:06:23.327873 10525 webserver.cc:460] Webserver started at http://127.25.130.1:38213/ using document root <none> and password file <none>
I20260812 19:06:23.328480 10525 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:23.328555 10525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:23.328787 10525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:06:23.331037 10525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/data/instance:
uuid: "7402118b06af4bb1bc693820faf92a6c"
format_stamp: "Formatted at 2026-08-12 19:06:23 on dist-test-slave-49h3"
server_key: "ee20f51d119e3aa2ff4faaa780658c08"
server_key_iv: "a667981a3b7f9c6654d32065cc6e2c77"
server_key_version: "encryptionkey@0"
I20260812 19:06:23.331638 10525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance:
uuid: "7402118b06af4bb1bc693820faf92a6c"
format_stamp: "Formatted at 2026-08-12 19:06:23 on dist-test-slave-49h3"
server_key: "ee20f51d119e3aa2ff4faaa780658c08"
server_key_iv: "a667981a3b7f9c6654d32065cc6e2c77"
server_key_version: "encryptionkey@0"
I20260812 19:06:23.335753 10525 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:06:23.338446 10540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:23.339818 10525 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:06:23.339952 10525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "7402118b06af4bb1bc693820faf92a6c"
format_stamp: "Formatted at 2026-08-12 19:06:23 on dist-test-slave-49h3"
server_key: "ee20f51d119e3aa2ff4faaa780658c08"
server_key_iv: "a667981a3b7f9c6654d32065cc6e2c77"
server_key_version: "encryptionkey@0"
I20260812 19:06:23.340075 10525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:23.360985 10525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:23.364303 10525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:06:23.364502 10525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:23.365156 10525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:06:23.366118 10525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:06:23.366170 10525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:23.366216 10525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:06:23.366246 10525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:23.375286 10525 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:32921
I20260812 19:06:23.375310 10676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:32921 every 8 connection(s)
I20260812 19:06:23.376284 10525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:06:23.381208 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 10525
I20260812 19:06:23.381340 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance
I20260812 19:06:23.381650 26120 external_mini_cluster.cc:1428] Setting key c40adf373bb41088d565808daa4fa622
Aug 12 19:06:23 dist-test-slave-49h3 krb5kdc[10388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561583, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:06:23.391194 10496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:23.378228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:49395 (local address 127.25.130.62:43125)
0812 19:06:23.378357 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:23.378361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:23.379209 (+ 848us) server_negotiation.cc:407] Connection header received
0812 19:06:23.380665 (+ 1456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:23.380669 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:23.380726 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:23.380801 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:23.382288 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:23.382958 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:23.384195 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:23.384510 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:23.384615 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:23.387247 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:23.387251 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:23.387254 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:23.387284 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:23.388898 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:23.389431 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:23.389435 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:23.389437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:23.389565 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:23.389961 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:23.389965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:23.389967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:23.390225 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:23.390325 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:23.390862 (+ 537us) server_negotiation.cc:299] Negotiation successful
0812 19:06:23.391012 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 19:06:23.392027 10679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:23.378548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.1:49395)
0812 19:06:23.379042 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:23.379095 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:23.380385 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:23.380983 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:23.380990 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:23.381397 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:06:23.382126 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:23.382148 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:23.383096 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:23.383100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:23.384053 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:23.384064 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:23.384647 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:23.384649 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:23.384797 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:23.385404 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:06:23.385422 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:06:23.387056 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:06:23.389025 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:23.389031 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:23.389049 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:23.389330 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:23.389695 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:23.389698 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:23.389700 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:23.389862 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:23.390325 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:06:23.390330 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:06:23.390609 (+ 279us) client_negotiation.cc:769] Sending connection context
0812 19:06:23.390779 (+ 170us) client_negotiation.cc:240] Negotiation successful
0812 19:06:23.391079 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":122,"threads_started":1}
WARNING: no policy specified for kudu/127.25.130.2@KRBTEST.COM; defaulting to no policy
I20260812 19:06:23.393401 10677 heartbeater.cc:347] Connected to a master server at 127.25.130.62:43125
I20260812 19:06:23.393671 10677 heartbeater.cc:464] Registering TS with master...
I20260812 19:06:23.394191 10677 heartbeater.cc:511] Master 127.25.130.62:43125 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.2@KRBTEST.COM" created.
I20260812 19:06:23.395794 10443 ts_manager.cc:194] Registered new tserver with Master: 7402118b06af4bb1bc693820faf92a6c (127.25.130.1:32921)
I20260812 19:06:23.396955 10443 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.1@KRBTEST.COM'} at 127.25.130.1:49395
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
I20260812 19:06:23.411973 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:0
--local_ip_for_outbound_sockets=127.25.130.2
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:43125
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:35319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:06:23.527427 10683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:06:23.527740 10683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:06:23.527784 10683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:06:23.531957 10683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:06:23.532037 10683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:06:23.532099 10683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:06:23.537606 10683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:35319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:43125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:06:23.538786 10683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:06:23.539834 10683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:23.549707 10693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:23.549702 10691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:23.549702 10690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:23.549844 10683 server_base.cc:1034] running on GCE node
I20260812 19:06:23.550439 10683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:06:23.551095 10683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:06:23.552280 10683 hybrid_clock.cc:648] HybridClock initialized: now 1786561583552257 us; error 38 us; skew 500 ppm
Aug 12 19:06:23 dist-test-slave-49h3 krb5kdc[10388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561583, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:23.555025 10683 init.cc:377] Logged in from keytab as kudu/127.25.130.2@KRBTEST.COM (short username kudu)
I20260812 19:06:23.555955 10683 webserver.cc:460] Webserver started at http://127.25.130.2:38827/ using document root <none> and password file <none>
I20260812 19:06:23.556514 10683 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:23.556574 10683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:23.556763 10683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:06:23.558970 10683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/data/instance:
uuid: "122a6c94061e4dc394286c9ca63162c4"
format_stamp: "Formatted at 2026-08-12 19:06:23 on dist-test-slave-49h3"
server_key: "88dc95548b82ec0854df976f5f3f910d"
server_key_iv: "d8a997efc64fad3f03f82471bfd77298"
server_key_version: "encryptionkey@0"
I20260812 19:06:23.559541 10683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance:
uuid: "122a6c94061e4dc394286c9ca63162c4"
format_stamp: "Formatted at 2026-08-12 19:06:23 on dist-test-slave-49h3"
server_key: "88dc95548b82ec0854df976f5f3f910d"
server_key_iv: "d8a997efc64fad3f03f82471bfd77298"
server_key_version: "encryptionkey@0"
I20260812 19:06:23.563319 10683 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260812 19:06:23.565727 10700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:23.566975 10683 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260812 19:06:23.567091 10683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "122a6c94061e4dc394286c9ca63162c4"
format_stamp: "Formatted at 2026-08-12 19:06:23 on dist-test-slave-49h3"
server_key: "88dc95548b82ec0854df976f5f3f910d"
server_key_iv: "d8a997efc64fad3f03f82471bfd77298"
server_key_version: "encryptionkey@0"
I20260812 19:06:23.567204 10683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:23.586170 10683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:23.589170 10683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:06:23.589383 10683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:23.590055 10683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:06:23.590929 10683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:06:23.590982 10683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:23.591037 10683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:06:23.591064 10683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:23.600075 10683 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:41435
I20260812 19:06:23.600103 10836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:41435 every 8 connection(s)
I20260812 19:06:23.601001 10683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:06:23.609913 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 10683
I20260812 19:06:23.610033 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance
Aug 12 19:06:23 dist-test-slave-49h3 krb5kdc[10388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561583, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:06:23.610337 26120 external_mini_cluster.cc:1428] Setting key a2f6bf7ea1a8c6227ef5bd457515bb27
I20260812 19:06:23.614670 10496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:23.602889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:55541 (local address 127.25.130.62:43125)
0812 19:06:23.603021 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:23.603025 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:23.604117 (+ 1092us) server_negotiation.cc:407] Connection header received
0812 19:06:23.604996 (+ 879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:23.604998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:23.605045 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:23.605116 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:23.606324 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:23.606854 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:23.607954 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:23.608178 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:23.608257 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:23.611033 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:23.611036 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:23.611038 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:23.611063 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:23.612541 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:23.613063 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:23.613067 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:23.613068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:23.613161 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:23.613473 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:23.613475 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:23.613477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:23.613698 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:23.613801 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:23.614368 (+ 567us) server_negotiation.cc:299] Negotiation successful
0812 19:06:23.614505 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 19:06:23.615494 10840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:23.603272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.2:55541)
0812 19:06:23.603964 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:23.604007 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:23.604818 (+ 811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:23.605266 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:23.605273 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:23.605695 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:06:23.606205 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:23.606219 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:23.606949 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:23.606951 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:23.607847 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:23.607858 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:23.608310 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:23.608312 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:23.608458 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:23.609105 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:06:23.609123 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:06:23.610861 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:06:23.612672 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:23.612678 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:23.612693 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:23.612967 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:23.613238 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:23.613240 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:23.613241 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:23.613373 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:23.613817 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:06:23.613823 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:06:23.614114 (+ 291us) client_negotiation.cc:769] Sending connection context
0812 19:06:23.614286 (+ 172us) client_negotiation.cc:240] Negotiation successful
0812 19:06:23.614566 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":119,"threads_started":1}
I20260812 19:06:23.616683 10837 heartbeater.cc:347] Connected to a master server at 127.25.130.62:43125
I20260812 19:06:23.616982 10837 heartbeater.cc:464] Registering TS with master...
I20260812 19:06:23.617667 10837 heartbeater.cc:511] Master 127.25.130.62:43125 requested a full tablet report, sending...
I20260812 19:06:23.618873 10443 ts_manager.cc:194] Registered new tserver with Master: 122a6c94061e4dc394286c9ca63162c4 (127.25.130.2:41435)
I20260812 19:06:23.619572 10443 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.2@KRBTEST.COM'} at 127.25.130.2:55541
WARNING: no policy specified for kudu/127.25.130.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
I20260812 19:06:23.638734 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:0
--local_ip_for_outbound_sockets=127.25.130.3
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:43125
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:35319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:06:23.753801 10844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:06:23.754086 10844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:06:23.754132 10844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:06:23.758309 10844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:06:23.758379 10844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:06:23.758451 10844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:06:23.763959 10844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:35319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:43125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:06:23.765198 10844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:06:23.766352 10844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:23.776346 10850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:23.776432 10853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:23.776401 10851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:23.776553 10844 server_base.cc:1034] running on GCE node
I20260812 19:06:23.777060 10844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:06:23.777796 10844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:06:23.778951 10844 hybrid_clock.cc:648] HybridClock initialized: now 1786561583778929 us; error 35 us; skew 500 ppm
Aug 12 19:06:23 dist-test-slave-49h3 krb5kdc[10388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561583, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:23.781787 10844 init.cc:377] Logged in from keytab as kudu/127.25.130.3@KRBTEST.COM (short username kudu)
I20260812 19:06:23.782747 10844 webserver.cc:460] Webserver started at http://127.25.130.3:45187/ using document root <none> and password file <none>
I20260812 19:06:23.783296 10844 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:23.783355 10844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:23.783532 10844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:06:23.785642 10844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/data/instance:
uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1"
format_stamp: "Formatted at 2026-08-12 19:06:23 on dist-test-slave-49h3"
server_key: "816763db7740ab739d34f87b6d5f4ee8"
server_key_iv: "b40bf501143fff9ea299f2e64312d603"
server_key_version: "encryptionkey@0"
I20260812 19:06:23.786222 10844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance:
uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1"
format_stamp: "Formatted at 2026-08-12 19:06:23 on dist-test-slave-49h3"
server_key: "816763db7740ab739d34f87b6d5f4ee8"
server_key_iv: "b40bf501143fff9ea299f2e64312d603"
server_key_version: "encryptionkey@0"
I20260812 19:06:23.790081 10844 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 19:06:23.792658 10861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:23.794054 10844 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:06:23.794180 10844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1"
format_stamp: "Formatted at 2026-08-12 19:06:23 on dist-test-slave-49h3"
server_key: "816763db7740ab739d34f87b6d5f4ee8"
server_key_iv: "b40bf501143fff9ea299f2e64312d603"
server_key_version: "encryptionkey@0"
I20260812 19:06:23.794313 10844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:23.817219 10844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:23.820601 10844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:06:23.820801 10844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:23.821430 10844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:06:23.822332 10844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:06:23.822388 10844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:23.822434 10844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:06:23.822463 10844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:23.831002 10844 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:46001
I20260812 19:06:23.831036 11002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:46001 every 8 connection(s)
I20260812 19:06:23.831933 10844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:06:23.836514 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 10844
I20260812 19:06:23.836629 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance
I20260812 19:06:23.836875 26120 external_mini_cluster.cc:1428] Setting key ab4d49f15d6a8159b71ed251477564c2
Aug 12 19:06:23 dist-test-slave-49h3 krb5kdc[10388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561583, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:06:23.845932 10496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:23.833838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:33873 (local address 127.25.130.62:43125)
0812 19:06:23.834004 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:23.834009 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:23.834906 (+ 897us) server_negotiation.cc:407] Connection header received
0812 19:06:23.836227 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:23.836232 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:23.836295 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:23.836390 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:23.837801 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:23.838469 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:23.839773 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:23.839990 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:23.840063 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:23.842505 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:23.842509 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:23.842511 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:23.842536 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:23.843874 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:23.844371 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:23.844373 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:23.844375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:23.844444 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:23.844818 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:23.844820 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:23.844821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:23.845015 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:23.845090 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:23.845697 (+ 607us) server_negotiation.cc:299] Negotiation successful
0812 19:06:23.845806 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 19:06:23.846882 11006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:23.834262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.3:33873)
0812 19:06:23.834749 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:23.834796 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:23.836028 (+ 1232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:23.836554 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:23.836561 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:23.836987 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:06:23.837653 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:23.837675 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:23.838602 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:23.838606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:23.839656 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:23.839667 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:23.840152 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:23.840156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:23.840307 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:23.840842 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:06:23.840860 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:06:23.842339 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:06:23.843996 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:23.844003 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:23.844020 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:23.844293 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:23.844560 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:23.844563 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:23.844564 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:23.844737 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:23.845105 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:06:23.845111 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:06:23.845369 (+ 258us) client_negotiation.cc:769] Sending connection context
0812 19:06:23.845673 (+ 304us) client_negotiation.cc:240] Negotiation successful
0812 19:06:23.845991 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":120,"threads_started":1}
I20260812 19:06:23.848069 11004 heartbeater.cc:347] Connected to a master server at 127.25.130.62:43125
I20260812 19:06:23.848302 11004 heartbeater.cc:464] Registering TS with master...
I20260812 19:06:23.848804 11004 heartbeater.cc:511] Master 127.25.130.62:43125 requested a full tablet report, sending...
I20260812 19:06:23.849959 10443 ts_manager.cc:194] Registered new tserver with Master: 16ebfcb8b3dc4ab885bb6138969fc8f1 (127.25.130.3:46001)
I20260812 19:06:23.850692 10443 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.3@KRBTEST.COM'} at 127.25.130.3:33873
I20260812 19:06:23.850863 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:06:23.859606 10496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:23.853107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43368 (local address 127.25.130.62:43125)
0812 19:06:23.853255 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:23.853260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:23.853390 (+ 130us) server_negotiation.cc:407] Connection header received
0812 19:06:23.853528 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:23.853530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:23.853570 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:23.853631 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:23.854257 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:23.854748 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:23.855477 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:23.855674 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:23.855738 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:23.856938 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:23.856941 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:23.856944 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:23.856965 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:23.858296 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:23.858648 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:23.858650 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:23.858651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:23.858716 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:23.858902 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:23.858904 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:23.858905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:23.859106 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:23.859214 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:23.859358 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 19:06:23.859452 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 19:06:23.862298 10443 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43368:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:06:23.865270 10443 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:06:23.879755 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:23.874075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55266 (local address 127.25.130.3:46001)
0812 19:06:23.875458 (+ 1383us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:23.875468 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:23.875489 (+ 21us) server_negotiation.cc:407] Connection header received
0812 19:06:23.875556 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:23.875561 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:23.875728 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:23.876214 (+ 486us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:06:23.876342 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:23.877036 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:23.878521 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:23.879088 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:23.879184 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:23.879343 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:23.879491 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 19:06:23.879590 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1283,"thread_start_us":164,"threads_started":1}
I20260812 19:06:23.879741 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:23.873936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:46001 (local address 127.0.0.1:55266)
0812 19:06:23.874756 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:23.874807 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:23.874933 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:23.875873 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:23.875877 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:23.875910 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:06:23.876063 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:23.876070 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:23.877169 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:23.877172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:23.878400 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:23.878410 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:23.879202 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:23.879206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:23.879312 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:23.879349 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 19:06:23.879451 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 19:06:23.879586 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"mutex_wait_us":2,"spinlock_wait_cycles":7808,"thread_start_us":102,"threads_started":1}
I20260812 19:06:23.880273 11020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:23.874781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:41435 (local address 127.0.0.1:53614)
0812 19:06:23.875160 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:23.875171 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:23.875254 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:23.875875 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:23.875877 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:23.875911 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:06:23.876080 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:23.876087 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:23.877649 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:23.877652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:23.878796 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:23.878802 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:23.879893 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:23.879896 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:23.879992 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:23.880016 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 19:06:23.880063 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 19:06:23.880132 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":74,"spinlock_wait_cycles":11776,"thread_start_us":99,"threads_started":1}
I20260812 19:06:23.880529 11017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:23.874094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:32921 (local address 127.0.0.1:56722)
0812 19:06:23.875094 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:23.875108 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:23.875209 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:23.875757 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:23.875763 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:23.875787 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:06:23.875982 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:23.875991 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:23.877523 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:23.877526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:23.878788 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:23.878799 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:23.880057 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:23.880060 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:23.880151 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:23.880182 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 19:06:23.880244 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 19:06:23.880302 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":910,"mutex_wait_us":41,"thread_start_us":106,"threads_started":1}
I20260812 19:06:23.880913 11021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:23.875109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53614 (local address 127.25.130.2:41435)
0812 19:06:23.875469 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:23.875481 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:23.875503 (+ 22us) server_negotiation.cc:407] Connection header received
0812 19:06:23.875568 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:23.875571 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:23.875731 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:23.876012 (+ 281us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:06:23.876184 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:23.877523 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:23.878938 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:23.879757 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:23.880462 (+ 705us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:23.880611 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:23.880662 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 19:06:23.880767 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":123,"threads_started":1}
I20260812 19:06:23.881240 11019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:23.874289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56722 (local address 127.25.130.1:32921)
0812 19:06:23.874711 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:23.874729 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:23.875191 (+ 462us) server_negotiation.cc:407] Connection header received
0812 19:06:23.875351 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:23.875356 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:23.875587 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:23.875746 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:06:23.876123 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:23.877339 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:23.878931 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:23.879755 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:23.879859 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:23.880000 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:23.880984 (+ 984us) server_negotiation.cc:299] Negotiation successful
0812 19:06:23.881086 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":60,"threads_started":1}
I20260812 19:06:23.883325 10922 tablet_service.cc:1467] Processing CreateTablet for tablet c05432e1a79441ed922b371137d090a4 (DEFAULT_TABLE table=test-table [id=87e06467b967474c8dbe760d09071c4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:06:23.883595 10599 tablet_service.cc:1467] Processing CreateTablet for tablet c05432e1a79441ed922b371137d090a4 (DEFAULT_TABLE table=test-table [id=87e06467b967474c8dbe760d09071c4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:06:23.883693 10757 tablet_service.cc:1467] Processing CreateTablet for tablet c05432e1a79441ed922b371137d090a4 (DEFAULT_TABLE table=test-table [id=87e06467b967474c8dbe760d09071c4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:06:23.884508 10599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c05432e1a79441ed922b371137d090a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:23.884534 10922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c05432e1a79441ed922b371137d090a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:23.884534 10757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c05432e1a79441ed922b371137d090a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:23.890879 11022 tablet_bootstrap.cc:492] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c: Bootstrap starting.
I20260812 19:06:23.892475 11025 tablet_bootstrap.cc:492] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4: Bootstrap starting.
I20260812 19:06:23.893194 11022 tablet_bootstrap.cc:654] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:23.893792 11024 tablet_bootstrap.cc:492] T c05432e1a79441ed922b371137d090a4 P 16ebfcb8b3dc4ab885bb6138969fc8f1: Bootstrap starting.
I20260812 19:06:23.894025 11022 log.cc:824] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c: Log is configured to *not* fsync() on all Append() calls
I20260812 19:06:23.895032 11025 tablet_bootstrap.cc:654] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:23.895896 11025 log.cc:824] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4: Log is configured to *not* fsync() on all Append() calls
I20260812 19:06:23.896090 11022 tablet_bootstrap.cc:492] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c: No bootstrap required, opened a new log
I20260812 19:06:23.896327 11022 ts_tablet_manager.cc:1397] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 19:06:23.896435 11024 tablet_bootstrap.cc:654] T c05432e1a79441ed922b371137d090a4 P 16ebfcb8b3dc4ab885bb6138969fc8f1: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:23.897207 11024 log.cc:824] T c05432e1a79441ed922b371137d090a4 P 16ebfcb8b3dc4ab885bb6138969fc8f1: Log is configured to *not* fsync() on all Append() calls
I20260812 19:06:23.898270 11025 tablet_bootstrap.cc:492] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4: No bootstrap required, opened a new log
I20260812 19:06:23.898483 11025 ts_tablet_manager.cc:1397] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 19:06:23.898959 11024 tablet_bootstrap.cc:492] T c05432e1a79441ed922b371137d090a4 P 16ebfcb8b3dc4ab885bb6138969fc8f1: No bootstrap required, opened a new log
I20260812 19:06:23.899166 11024 ts_tablet_manager.cc:1397] T c05432e1a79441ed922b371137d090a4 P 16ebfcb8b3dc4ab885bb6138969fc8f1: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20260812 19:06:23.900575 11022 raft_consensus.cc:348] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } }
I20260812 19:06:23.900874 11022 raft_consensus.cc:374] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:23.900969 11022 raft_consensus.cc:729] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7402118b06af4bb1bc693820faf92a6c, State: Initialized, Role: FOLLOWER
I20260812 19:06:23.901494 11022 consensus_queue.cc:260] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } }
I20260812 19:06:23.902304 10677 heartbeater.cc:503] Master 127.25.130.62:43125 was elected leader, sending a full tablet report...
I20260812 19:06:23.902294 11024 raft_consensus.cc:348] T c05432e1a79441ed922b371137d090a4 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } }
I20260812 19:06:23.902611 11022 ts_tablet_manager.cc:1428] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 19:06:23.902612 11024 raft_consensus.cc:374] T c05432e1a79441ed922b371137d090a4 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:23.902673 11024 raft_consensus.cc:729] T c05432e1a79441ed922b371137d090a4 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16ebfcb8b3dc4ab885bb6138969fc8f1, State: Initialized, Role: FOLLOWER
I20260812 19:06:23.903098 11024 consensus_queue.cc:260] T c05432e1a79441ed922b371137d090a4 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } }
I20260812 19:06:23.903463 11025 raft_consensus.cc:348] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } }
I20260812 19:06:23.903729 11004 heartbeater.cc:503] Master 127.25.130.62:43125 was elected leader, sending a full tablet report...
I20260812 19:06:23.903759 11025 raft_consensus.cc:374] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:23.903816 11025 raft_consensus.cc:729] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 122a6c94061e4dc394286c9ca63162c4, State: Initialized, Role: FOLLOWER
I20260812 19:06:23.904040 11024 ts_tablet_manager.cc:1428] T c05432e1a79441ed922b371137d090a4 P 16ebfcb8b3dc4ab885bb6138969fc8f1: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260812 19:06:23.904268 11025 consensus_queue.cc:260] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } }
I20260812 19:06:23.904982 10837 heartbeater.cc:503] Master 127.25.130.62:43125 was elected leader, sending a full tablet report...
I20260812 19:06:23.905215 11025 ts_tablet_manager.cc:1428] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260812 19:06:24.082924 11005 tablet.cc:2369] T c05432e1a79441ed922b371137d090a4 P 16ebfcb8b3dc4ab885bb6138969fc8f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:06:24.090121 11033 raft_consensus.cc:482] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:24.090359 11033 raft_consensus.cc:504] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } }
I20260812 19:06:24.091612 11033 leader_election.cc:288] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16ebfcb8b3dc4ab885bb6138969fc8f1 (127.25.130.3:46001), 7402118b06af4bb1bc693820faf92a6c (127.25.130.1:32921)
I20260812 19:06:24.095692 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:24.092007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:60253 (local address 127.25.130.3:46001)
0812 19:06:24.092133 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:24.092138 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:24.092483 (+ 345us) server_negotiation.cc:407] Connection header received
0812 19:06:24.092578 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:24.092582 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:24.092634 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:24.092711 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:06:24.092992 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:24.093522 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:24.094660 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:24.095067 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:24.095173 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:24.095226 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:24.095476 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 19:06:24.095552 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 19:06:24.095692 11019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:24.092036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:33313 (local address 127.25.130.1:32921)
0812 19:06:24.092177 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:24.092181 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:24.092287 (+ 106us) server_negotiation.cc:407] Connection header received
0812 19:06:24.092448 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:24.092452 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:24.092506 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:24.092583 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:06:24.092864 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:24.093511 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:24.094618 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:24.095048 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:24.095156 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:24.095215 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:24.095470 (+ 255us) server_negotiation.cc:299] Negotiation successful
0812 19:06:24.095551 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 19:06:24.095692 10840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:24.091928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:32921 (local address 127.25.130.2:33313)
0812 19:06:24.092193 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:24.092209 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:24.092333 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:24.092594 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:24.092597 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:24.092613 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:06:24.092748 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:24.092755 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:24.093610 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:24.093613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:24.094492 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:24.094503 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:24.095168 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:24.095172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:24.095352 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:24.095372 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 19:06:24.095439 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 19:06:24.095564 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":90}
I20260812 19:06:24.095705 11036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:24.091930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:46001 (local address 127.25.130.2:60253)
0812 19:06:24.092370 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:24.092384 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:24.092490 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:24.092749 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:24.092751 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:24.092766 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:06:24.092882 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:24.092889 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:24.093629 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:24.093632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:24.094522 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:24.094533 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:24.095176 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:24.095180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:24.095357 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:24.095375 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 19:06:24.095438 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 19:06:24.095551 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"mutex_wait_us":150,"thread_start_us":148,"threads_started":1}
I20260812 19:06:24.096474 10623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c05432e1a79441ed922b371137d090a4" candidate_uuid: "122a6c94061e4dc394286c9ca63162c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7402118b06af4bb1bc693820faf92a6c" is_pre_election: true
I20260812 19:06:24.096539 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c05432e1a79441ed922b371137d090a4" candidate_uuid: "122a6c94061e4dc394286c9ca63162c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" is_pre_election: true
I20260812 19:06:24.096815 10623 raft_consensus.cc:2454] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 122a6c94061e4dc394286c9ca63162c4 in term 0.
I20260812 19:06:24.096829 10942 raft_consensus.cc:2454] T c05432e1a79441ed922b371137d090a4 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 122a6c94061e4dc394286c9ca63162c4 in term 0.
I20260812 19:06:24.097293 10701 leader_election.cc:302] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 122a6c94061e4dc394286c9ca63162c4, 7402118b06af4bb1bc693820faf92a6c; no voters:
I20260812 19:06:24.097644 11033 raft_consensus.cc:2781] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:24.097738 11033 raft_consensus.cc:482] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:24.097785 11033 raft_consensus.cc:3037] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:24.098949 11033 raft_consensus.cc:504] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } }
I20260812 19:06:24.099304 11033 leader_election.cc:288] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4 [CANDIDATE]: Term 1 election: Requested vote from peers 16ebfcb8b3dc4ab885bb6138969fc8f1 (127.25.130.3:46001), 7402118b06af4bb1bc693820faf92a6c (127.25.130.1:32921)
I20260812 19:06:24.099711 10623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c05432e1a79441ed922b371137d090a4" candidate_uuid: "122a6c94061e4dc394286c9ca63162c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7402118b06af4bb1bc693820faf92a6c"
I20260812 19:06:24.099711 10942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c05432e1a79441ed922b371137d090a4" candidate_uuid: "122a6c94061e4dc394286c9ca63162c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1"
I20260812 19:06:24.099856 10623 raft_consensus.cc:3037] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:24.099855 10942 raft_consensus.cc:3037] T c05432e1a79441ed922b371137d090a4 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:24.101188 10623 raft_consensus.cc:2454] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 122a6c94061e4dc394286c9ca63162c4 in term 1.
I20260812 19:06:24.101188 10942 raft_consensus.cc:2454] T c05432e1a79441ed922b371137d090a4 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 122a6c94061e4dc394286c9ca63162c4 in term 1.
I20260812 19:06:24.101572 10704 leader_election.cc:302] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 122a6c94061e4dc394286c9ca63162c4, 16ebfcb8b3dc4ab885bb6138969fc8f1; no voters:
I20260812 19:06:24.101764 11033 raft_consensus.cc:2781] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:24.102025 11033 raft_consensus.cc:686] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4 [term 1 LEADER]: Becoming Leader. State: Replica: 122a6c94061e4dc394286c9ca63162c4, State: Running, Role: LEADER
W20260812 19:06:24.102162 10839 tablet.cc:2369] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:06:24.102435 11033 consensus_queue.cc:237] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } }
I20260812 19:06:24.105580 10442 catalog_manager.cc:5495] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 122a6c94061e4dc394286c9ca63162c4 (127.25.130.2). New cstate: current_term: 1 leader_uuid: "122a6c94061e4dc394286c9ca63162c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } health_report { overall_health: HEALTHY } } }
W20260812 19:06:24.127725 10678 tablet.cc:2369] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:06:24.172441 11021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:24.168759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53620 (local address 127.25.130.2:41435)
0812 19:06:24.168897 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:24.168901 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:24.168917 (+ 16us) server_negotiation.cc:407] Connection header received
0812 19:06:24.169005 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:24.169008 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:24.169073 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:24.169173 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:06:24.169466 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:24.170000 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:24.170870 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:24.171134 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:24.171228 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:24.171564 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:06:24.172005 (+ 441us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:06:24.172128 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:24.172196 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 19:06:24.172263 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 19:06:24.178931 10623 raft_consensus.cc:1261] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c [term 1 FOLLOWER]: Refusing update from remote peer 122a6c94061e4dc394286c9ca63162c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:24.178934 10942 raft_consensus.cc:1261] T c05432e1a79441ed922b371137d090a4 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [term 1 FOLLOWER]: Refusing update from remote peer 122a6c94061e4dc394286c9ca63162c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:24.179673 11033 consensus_queue.cc:1035] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:24.179884 11037 consensus_queue.cc:1035] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:24.186908 11040 mvcc.cc:205] Tried to move back new op lower bound from 7317756248788488192 to 7317756248486563840. Current Snapshot: MvccSnapshot[applied={T|T < 7317756248788488192}]
I20260812 19:06:24.187856 11041 mvcc.cc:205] Tried to move back new op lower bound from 7317756248788488192 to 7317756248486563840. Current Snapshot: MvccSnapshot[applied={T|T < 7317756248788488192}]
I20260812 19:06:24.191071 11042 mvcc.cc:205] Tried to move back new op lower bound from 7317756248788488192 to 7317756248486563840. Current Snapshot: MvccSnapshot[applied={T|T < 7317756248788488192 or (T in {7317756248788488192})}]
I20260812 19:06:24.196462 10443 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:06:24.196681 10443 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:06:24.199343 10443 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7: Sending DeleteTablet for 3 replicas of tablet c05432e1a79441ed922b371137d090a4
I20260812 19:06:24.200294 10757 tablet_service.cc:1514] Processing DeleteTablet for tablet c05432e1a79441ed922b371137d090a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:06:24 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:53614
I20260812 19:06:24.200274 10599 tablet_service.cc:1514] Processing DeleteTablet for tablet c05432e1a79441ed922b371137d090a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:06:24 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:56722
I20260812 19:06:24.200549 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 10525
I20260812 19:06:24.200868 10922 tablet_service.cc:1514] Processing DeleteTablet for tablet c05432e1a79441ed922b371137d090a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:06:24 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:55266
I20260812 19:06:24.203276 11051 tablet_replica.cc:330] T c05432e1a79441ed922b371137d090a4 P 16ebfcb8b3dc4ab885bb6138969fc8f1: stopping tablet replica
I20260812 19:06:24.203666 11051 raft_consensus.cc:2229] T c05432e1a79441ed922b371137d090a4 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:24.204027 11051 raft_consensus.cc:2258] T c05432e1a79441ed922b371137d090a4 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:24.207350 11049 tablet_replica.cc:330] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4: stopping tablet replica
I20260812 19:06:24.207680 11051 ts_tablet_manager.cc:1905] T c05432e1a79441ed922b371137d090a4 P 16ebfcb8b3dc4ab885bb6138969fc8f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:06:24.207738 11049 raft_consensus.cc:2229] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4 [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:06:24.208204 11049 raft_consensus.cc:2258] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:24.209564 11049 ts_tablet_manager.cc:1905] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:06:24.210881 11051 ts_tablet_manager.cc:1918] T c05432e1a79441ed922b371137d090a4 P 16ebfcb8b3dc4ab885bb6138969fc8f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:06:24.211045 11051 log.cc:1196] T c05432e1a79441ed922b371137d090a4 P 16ebfcb8b3dc4ab885bb6138969fc8f1: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/wal/wals/c05432e1a79441ed922b371137d090a4
I20260812 19:06:24.211478 11051 ts_tablet_manager.cc:1939] T c05432e1a79441ed922b371137d090a4 P 16ebfcb8b3dc4ab885bb6138969fc8f1: Deleting consensus metadata
I20260812 19:06:24.212348 11049 ts_tablet_manager.cc:1918] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:06:24.212484 11049 log.cc:1196] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/wal/wals/c05432e1a79441ed922b371137d090a4
I20260812 19:06:24.212832 11049 ts_tablet_manager.cc:1939] T c05432e1a79441ed922b371137d090a4 P 122a6c94061e4dc394286c9ca63162c4: Deleting consensus metadata
I20260812 19:06:24.213109 10428 catalog_manager.cc:4841] TS 16ebfcb8b3dc4ab885bb6138969fc8f1 (127.25.130.3:46001): tablet c05432e1a79441ed922b371137d090a4 (table test-table [id=87e06467b967474c8dbe760d09071c4c]) successfully deleted
I20260812 19:06:24.214021 10426 catalog_manager.cc:4841] TS 122a6c94061e4dc394286c9ca63162c4 (127.25.130.2:41435): tablet c05432e1a79441ed922b371137d090a4 (table test-table [id=87e06467b967474c8dbe760d09071c4c]) successfully deleted
W20260812 19:06:24.214128 10425 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.130.1:32921 (error 108)
I20260812 19:06:24.214403 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 10683
I20260812 19:06:24.220396 11017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:24.220038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:32921 (local address 127.0.0.1:56730)
0812 19:06:24.220210 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:24.220302 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.1:32921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260812 19:06:24.220604 10425 catalog_manager.cc:4584] TS 7402118b06af4bb1bc693820faf92a6c (127.25.130.1:32921): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c05432e1a79441ed922b371137d090a4: Network error: Client connection negotiation failed: client connection to 127.25.130.1:32921: connect: Connection refused (error 111)
I20260812 19:06:24.224745 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 10844
I20260812 19:06:24.235329 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 10404
I20260812 19:06:24.244655 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:43125
--webserver_interface=127.25.130.62
--webserver_port=46101
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:35319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:43125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:06:24.359283 11053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:06:24.359577 11053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:06:24.359623 11053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:06:24.363780 11053 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:06:24.363857 11053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:06:24.363875 11053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:06:24.363893 11053 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:06:24.363924 11053 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:06:24.369575 11053 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:35319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:43125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:43125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=46101
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:06:24.370785 11053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:06:24.371886 11053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:24.386086 11064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:24.386260 11053 server_base.cc:1034] running on GCE node
W20260812 19:06:24.386106 11061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:24.386453 11067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:24.387068 11053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:06:24.388295 11053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:06:24.389482 11053 hybrid_clock.cc:648] HybridClock initialized: now 1786561584389457 us; error 33 us; skew 500 ppm
Aug 12 19:06:24 dist-test-slave-49h3 krb5kdc[10388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561584, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:24.392323 11053 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:06:24.393344 11053 webserver.cc:460] Webserver started at http://127.25.130.62:46101/ using document root <none> and password file <none>
I20260812 19:06:24.393980 11053 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:24.394066 11053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:24.398325 11053 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 19:06:24.400578 11078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:24.402086 11053 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:06:24.402226 11053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "e6b8bf975275492ab331eb132c0d92b7"
format_stamp: "Formatted at 2026-08-12 19:06:23 on dist-test-slave-49h3"
server_key: "e871a9b3f2cd887aeb459bca1de2a578"
server_key_iv: "f64f5fb3e4733f5f5f1b25e46c08aa5a"
server_key_version: "encryptionkey@0"
I20260812 19:06:24.402951 11053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:24.424943 11053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:24.428370 11053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:06:24.428589 11053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:24.436240 11053 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:43125
I20260812 19:06:24.436491 11134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:43125 every 8 connection(s)
I20260812 19:06:24.437487 11053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:06:24.441809 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 11053
I20260812 19:06:24.442179 11135 sys_catalog.cc:263] Verifying existing consensus state
I20260812 19:06:24.442260 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:32921
--local_ip_for_outbound_sockets=127.25.130.1
--tserver_master_addrs=127.25.130.62:43125
--webserver_port=38213
--webserver_interface=127.25.130.1
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:35319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:06:24.444171 11135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7: Bootstrap starting.
I20260812 19:06:24.468616 11135 log.cc:824] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7: Log is configured to *not* fsync() on all Append() calls
I20260812 19:06:24.480072 11135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:06:24.480545 11135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7: Bootstrap complete.
I20260812 19:06:24.486263 11135 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b8bf975275492ab331eb132c0d92b7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 43125 } }
I20260812 19:06:24.486850 11135 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b8bf975275492ab331eb132c0d92b7, State: Initialized, Role: FOLLOWER
I20260812 19:06:24.487444 11135 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b8bf975275492ab331eb132c0d92b7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 43125 } }
I20260812 19:06:24.487601 11135 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:06:24.487679 11135 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:06:24.487790 11135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:24.490525 11135 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b8bf975275492ab331eb132c0d92b7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 43125 } }
I20260812 19:06:24.490931 11135 leader_election.cc:302] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6b8bf975275492ab331eb132c0d92b7; no voters:
I20260812 19:06:24.491297 11135 leader_election.cc:288] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 19:06:24.491431 11139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:06:24.491680 11139 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [term 2 LEADER]: Becoming Leader. State: Replica: e6b8bf975275492ab331eb132c0d92b7, State: Running, Role: LEADER
I20260812 19:06:24.492040 11139 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b8bf975275492ab331eb132c0d92b7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 43125 } }
I20260812 19:06:24.492640 11135 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:06:24.493358 11140 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e6b8bf975275492ab331eb132c0d92b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b8bf975275492ab331eb132c0d92b7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 43125 } } }
I20260812 19:06:24.493569 11140 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [sys.catalog]: This master's current role is: LEADER
I20260812 19:06:24.494024 11142 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6b8bf975275492ab331eb132c0d92b7. Latest consensus state: current_term: 2 leader_uuid: "e6b8bf975275492ab331eb132c0d92b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b8bf975275492ab331eb132c0d92b7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 43125 } } }
I20260812 19:06:24.494117 11142 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [sys.catalog]: This master's current role is: LEADER
I20260812 19:06:24.495115 11149 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:06:24.500032 11149 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:06:24.500706 11149 catalog_manager.cc:1259] Loaded cluster ID: dab9e87dc88d41f4b59fc9335882d6f0
I20260812 19:06:24.500770 11149 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:06:24.502236 11149 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 19:06:24.507004 11149 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 19:06:24.512883 11149 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:06:24.513863 11149 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7: Loaded TSK: 0
I20260812 19:06:24.515125 11149 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 19:06:24.575505 11137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:06:24.575811 11137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:06:24.575877 11137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:06:24.580078 11137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:06:24.580148 11137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:06:24.580240 11137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:06:24.585815 11137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:35319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:32921
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=38213
--tserver_master_addrs=127.25.130.62:43125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:06:24.586944 11137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:06:24.588007 11137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:24.598785 11168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:24.598748 11169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:24.598762 11172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:24.599339 11137 server_base.cc:1034] running on GCE node
I20260812 19:06:24.599870 11137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:06:24.600741 11137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:06:24.601956 11137 hybrid_clock.cc:648] HybridClock initialized: now 1786561584601930 us; error 40 us; skew 500 ppm
Aug 12 19:06:24 dist-test-slave-49h3 krb5kdc[10388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561584, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:24.605593 11137 init.cc:377] Logged in from keytab as kudu/127.25.130.1@KRBTEST.COM (short username kudu)
I20260812 19:06:24.606866 11137 webserver.cc:460] Webserver started at http://127.25.130.1:38213/ using document root <none> and password file <none>
I20260812 19:06:24.607646 11137 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:24.607769 11137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:24.613324 11137 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 19:06:24.616182 11183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:24.617867 11137 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:06:24.618021 11137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "7402118b06af4bb1bc693820faf92a6c"
format_stamp: "Formatted at 2026-08-12 19:06:23 on dist-test-slave-49h3"
server_key: "ee20f51d119e3aa2ff4faaa780658c08"
server_key_iv: "a667981a3b7f9c6654d32065cc6e2c77"
server_key_version: "encryptionkey@0"
I20260812 19:06:24.618822 11137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:24.632836 11137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:24.636626 11137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:06:24.636837 11137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:24.637599 11137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:06:24.639009 11191 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 19:06:24.643522 11137 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 19:06:24.643617 11137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.001s
I20260812 19:06:24.643683 11137 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 19:06:24.645920 11137 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 19:06:24.645989 11137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260812 19:06:24.646598 11191 tablet_bootstrap.cc:492] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c: Bootstrap starting.
I20260812 19:06:24.656564 11137 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:32921
I20260812 19:06:24.656632 11316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:32921 every 8 connection(s)
I20260812 19:06:24.657661 11137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:06:24.661134 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 11137
I20260812 19:06:24.661674 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:41435
--local_ip_for_outbound_sockets=127.25.130.2
--tserver_master_addrs=127.25.130.62:43125
--webserver_port=38827
--webserver_interface=127.25.130.2
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:35319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:06:24.668426 11191 log.cc:824] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c: Log is configured to *not* fsync() on all Append() calls
Aug 12 19:06:24 dist-test-slave-49h3 krb5kdc[10388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561584, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:06:24.676448 11320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:24.659729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:42471 (local address 127.25.130.62:43125)
0812 19:06:24.660168 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:24.660191 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:24.660486 (+ 295us) server_negotiation.cc:407] Connection header received
0812 19:06:24.661863 (+ 1377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:24.661879 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:24.662318 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:24.662761 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:24.664071 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:24.665067 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:24.666306 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:24.666782 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:24.666947 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:24.670187 (+ 3240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:24.670196 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:24.670216 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:24.670269 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:24.672512 (+ 2243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:24.673269 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:24.673276 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:24.673281 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:24.673475 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:24.673914 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:24.673919 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:24.673921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:24.674476 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:24.674618 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:24.675287 (+ 669us) server_negotiation.cc:299] Negotiation successful
0812 19:06:24.675609 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":139,"threads_started":1}
I20260812 19:06:24.676467 11319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:24.659623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.1:42471)
0812 19:06:24.660310 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:24.660366 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:24.661359 (+ 993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:24.662845 (+ 1486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:24.662854 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:24.663350 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:06:24.663889 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:24.663905 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:24.665232 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:24.665235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:24.666171 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:24.666183 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:24.666936 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:24.666940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:24.667135 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:24.667850 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:06:24.667873 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:06:24.669899 (+ 2026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:06:24.672688 (+ 2789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:24.672696 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:24.672711 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:24.673101 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:24.673597 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:24.673600 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:24.673603 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:24.673801 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:24.674607 (+ 806us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:06:24.674613 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:06:24.674919 (+ 306us) client_negotiation.cc:769] Sending connection context
0812 19:06:24.675113 (+ 194us) client_negotiation.cc:240] Negotiation successful
0812 19:06:24.675483 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":147,"threads_started":1}
I20260812 19:06:24.676916 11191 tablet_bootstrap.cc:492] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:06:24.677379 11191 tablet_bootstrap.cc:492] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c: Bootstrap complete.
I20260812 19:06:24.678574 11191 ts_tablet_manager.cc:1397] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c: Time spent bootstrapping tablet: real 0.032s user 0.018s sys 0.011s
I20260812 19:06:24.679437 11317 heartbeater.cc:347] Connected to a master server at 127.25.130.62:43125
I20260812 19:06:24.679677 11317 heartbeater.cc:464] Registering TS with master...
I20260812 19:06:24.680233 11317 heartbeater.cc:511] Master 127.25.130.62:43125 requested a full tablet report, sending...
I20260812 19:06:24.681264 11191 raft_consensus.cc:348] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } }
I20260812 19:06:24.681766 11191 raft_consensus.cc:729] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7402118b06af4bb1bc693820faf92a6c, State: Initialized, Role: FOLLOWER
I20260812 19:06:24.682260 11191 consensus_queue.cc:260] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } }
I20260812 19:06:24.683112 11095 ts_manager.cc:194] Registered new tserver with Master: 7402118b06af4bb1bc693820faf92a6c (127.25.130.1:32921)
I20260812 19:06:24.683352 11191 ts_tablet_manager.cc:1428] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 19:06:24.686560 11095 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.1@KRBTEST.COM'} at 127.25.130.1:42471
I20260812 19:06:24.689271 11317 heartbeater.cc:503] Master 127.25.130.62:43125 was elected leader, sending a full tablet report...
Aug 12 19:06:24 dist-test-slave-49h3 krb5kdc[10388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561584, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for kudu/127.25.130.1@KRBTEST.COM
I20260812 19:06:24.696771 11326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:24.686131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:32921 (local address 127.0.0.1:56742)
0812 19:06:24.686550 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:24.686599 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:24.686734 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:24.687560 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:24.687567 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:24.687917 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:06:24.688032 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:24.688038 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:24.689001 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:24.689005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:24.689730 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:24.689742 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:24.690475 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:24.690480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:24.690595 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:24.691230 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:06:24.691243 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:06:24.692809 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:06:24.695075 (+ 2266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:24.695091 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:24.695095 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:24.695339 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:24.695627 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:24.695629 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:24.695631 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:24.695713 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:24.696216 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:06:24.696221 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:06:24.696337 (+ 116us) client_negotiation.cc:769] Sending connection context
0812 19:06:24.696451 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 19:06:24.696617 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":129,"threads_started":1}
I20260812 19:06:24.696888 11327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:24.686206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56742 (local address 127.25.130.1:32921)
0812 19:06:24.687002 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:24.687014 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:24.687039 (+ 25us) server_negotiation.cc:407] Connection header received
0812 19:06:24.687111 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:24.687116 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:24.687327 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:24.687456 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:24.688158 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:24.688883 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:24.689844 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:24.690342 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:24.690450 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:24.692953 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:24.692963 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:24.692976 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:24.693004 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:24.694940 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:24.695445 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:24.695451 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:24.695453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:24.695551 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:24.695784 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:24.695787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:24.695788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:24.696097 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:24.696230 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:24.696544 (+ 314us) server_negotiation.cc:299] Negotiation successful
0812 19:06:24.696724 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":691,"thread_start_us":92,"threads_started":1}
I20260812 19:06:24.697948 11239 tablet_service.cc:1514] Processing DeleteTablet for tablet c05432e1a79441ed922b371137d090a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:06:24 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:56742
I20260812 19:06:24.698947 11331 tablet_replica.cc:330] stopping tablet replica
I20260812 19:06:24.699239 11331 raft_consensus.cc:2229] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:24.699477 11331 raft_consensus.cc:2258] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:24.700433 11331 ts_tablet_manager.cc:1905] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:06:24.704195 11331 ts_tablet_manager.cc:1918] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:06:24.704303 11331 log.cc:1196] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/wal/wals/c05432e1a79441ed922b371137d090a4
I20260812 19:06:24.704541 11331 ts_tablet_manager.cc:1939] T c05432e1a79441ed922b371137d090a4 P 7402118b06af4bb1bc693820faf92a6c: Deleting consensus metadata
I20260812 19:06:24.706372 11079 catalog_manager.cc:4841] TS 7402118b06af4bb1bc693820faf92a6c (127.25.130.1:32921): tablet c05432e1a79441ed922b371137d090a4 (table test-table [id=87e06467b967474c8dbe760d09071c4c]) successfully deleted
W20260812 19:06:24.785838 11321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:06:24.786144 11321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:06:24.786190 11321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:06:24.790333 11321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:06:24.790407 11321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:06:24.790480 11321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:06:24.796316 11321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:35319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:41435
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=38827
--tserver_master_addrs=127.25.130.62:43125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:06:24.797430 11321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:06:24.798523 11321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:24.808992 11339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:24.808992 11338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:24.809166 11321 server_base.cc:1034] running on GCE node
W20260812 19:06:24.809409 11341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:24.809923 11321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:06:24.810640 11321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:06:24.811794 11321 hybrid_clock.cc:648] HybridClock initialized: now 1786561584811767 us; error 44 us; skew 500 ppm
Aug 12 19:06:24 dist-test-slave-49h3 krb5kdc[10388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561584, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:24.814716 11321 init.cc:377] Logged in from keytab as kudu/127.25.130.2@KRBTEST.COM (short username kudu)
I20260812 19:06:24.815687 11321 webserver.cc:460] Webserver started at http://127.25.130.2:38827/ using document root <none> and password file <none>
I20260812 19:06:24.816264 11321 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:24.816351 11321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:24.820442 11321 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 19:06:24.822685 11350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:24.824046 11321 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 19:06:24.824177 11321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "122a6c94061e4dc394286c9ca63162c4"
format_stamp: "Formatted at 2026-08-12 19:06:23 on dist-test-slave-49h3"
server_key: "88dc95548b82ec0854df976f5f3f910d"
server_key_iv: "d8a997efc64fad3f03f82471bfd77298"
server_key_version: "encryptionkey@0"
I20260812 19:06:24.824779 11321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:24.834928 11321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:24.837984 11321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:06:24.838181 11321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:24.838800 11321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:06:24.839691 11321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:06:24.839743 11321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:24.839798 11321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:06:24.839826 11321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:24.848884 11321 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:41435
I20260812 19:06:24.848910 11503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:41435 every 8 connection(s)
I20260812 19:06:24.849956 11321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:06:24.850272 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 11321
I20260812 19:06:24.850718 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:46001
--local_ip_for_outbound_sockets=127.25.130.3
--tserver_master_addrs=127.25.130.62:43125
--webserver_port=45187
--webserver_interface=127.25.130.3
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:35319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 19:06:24 dist-test-slave-49h3 krb5kdc[10388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561584, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:06:24.865187 11320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:24.852831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:60061 (local address 127.25.130.62:43125)
0812 19:06:24.852978 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:24.852983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:24.853351 (+ 368us) server_negotiation.cc:407] Connection header received
0812 19:06:24.854766 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:24.854769 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:24.854831 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:24.854931 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:24.856336 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:24.857159 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:24.858439 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:24.858801 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:24.858926 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:24.861412 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:24.861417 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:24.861420 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:24.861492 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:24.863077 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:24.863606 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:24.863609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:24.863611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:24.863702 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:24.864000 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:24.864003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:24.864005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:24.864239 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:24.864329 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:24.864853 (+ 524us) server_negotiation.cc:299] Negotiation successful
0812 19:06:24.865005 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 19:06:24.866086 11507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:24.852502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.2:60061)
0812 19:06:24.853181 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:24.853236 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:24.854521 (+ 1285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:24.855153 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:24.855161 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:24.855550 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:06:24.856142 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:24.856159 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:24.857312 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:24.857315 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:24.858302 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:24.858315 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:24.858955 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:24.858958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:24.859117 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:24.859716 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:06:24.859734 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:06:24.861219 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:06:24.863206 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:24.863212 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:24.863228 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:24.863514 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:24.863781 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:24.863783 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:24.863784 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:24.863928 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:24.864345 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:06:24.864351 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:06:24.864630 (+ 279us) client_negotiation.cc:769] Sending connection context
0812 19:06:24.864797 (+ 167us) client_negotiation.cc:240] Negotiation successful
0812 19:06:24.865082 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":138,"threads_started":1}
I20260812 19:06:24.867286 11504 heartbeater.cc:347] Connected to a master server at 127.25.130.62:43125
I20260812 19:06:24.867534 11504 heartbeater.cc:464] Registering TS with master...
I20260812 19:06:24.868052 11504 heartbeater.cc:511] Master 127.25.130.62:43125 requested a full tablet report, sending...
I20260812 19:06:24.869155 11095 ts_manager.cc:194] Registered new tserver with Master: 122a6c94061e4dc394286c9ca63162c4 (127.25.130.2:41435)
I20260812 19:06:24.869781 11095 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.2@KRBTEST.COM'} at 127.25.130.2:60061
W20260812 19:06:24.967537 11506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:06:24.967847 11506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:06:24.967911 11506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:06:24.972038 11506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:06:24.972132 11506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:06:24.972218 11506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:06:24.977792 11506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:35319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:46001
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=45187
--tserver_master_addrs=127.25.130.62:43125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:06:24.978957 11506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:06:24.980013 11506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:24.989463 11513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:24.989477 11512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:24.989477 11516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:24.989774 11506 server_base.cc:1034] running on GCE node
I20260812 19:06:24.990167 11506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:06:24.990767 11506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:06:24.991989 11506 hybrid_clock.cc:648] HybridClock initialized: now 1786561584991959 us; error 42 us; skew 500 ppm
Aug 12 19:06:24 dist-test-slave-49h3 krb5kdc[10388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561584, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:24.995190 11506 init.cc:377] Logged in from keytab as kudu/127.25.130.3@KRBTEST.COM (short username kudu)
I20260812 19:06:24.996138 11506 webserver.cc:460] Webserver started at http://127.25.130.3:45187/ using document root <none> and password file <none>
I20260812 19:06:24.996687 11506 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:24.996774 11506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:25.000815 11506 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 19:06:25.002969 11527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:25.004297 11506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 19:06:25.004417 11506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1"
format_stamp: "Formatted at 2026-08-12 19:06:23 on dist-test-slave-49h3"
server_key: "816763db7740ab739d34f87b6d5f4ee8"
server_key_iv: "b40bf501143fff9ea299f2e64312d603"
server_key_version: "encryptionkey@0"
I20260812 19:06:25.005021 11506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:25.023104 11506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:25.026532 11506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:06:25.026728 11506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:25.027359 11506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:06:25.028295 11506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:06:25.028348 11506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:25.028403 11506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:06:25.028429 11506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:25.037492 11506 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:46001
I20260812 19:06:25.037535 11665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:46001 every 8 connection(s)
I20260812 19:06:25.038545 11506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:06:25.039042 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 11506
I20260812 19:06:25.047798 11320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:25.039784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43384 (local address 127.25.130.62:43125)
0812 19:06:25.039937 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:25.039941 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:25.040231 (+ 290us) server_negotiation.cc:407] Connection header received
0812 19:06:25.040411 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:25.040414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:25.040457 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:25.040521 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:25.041262 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:25.041891 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:25.042762 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:25.043016 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:25.043132 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:25.044660 (+ 1528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:25.044664 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:25.044666 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:25.044691 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:25.046215 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:25.046683 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:25.046685 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:25.046686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:25.046758 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:25.047012 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:25.047015 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:25.047016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:25.047211 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:25.047420 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:25.047534 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 19:06:25.047643 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":53}
Aug 12 19:06:25 dist-test-slave-49h3 krb5kdc[10388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561584, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:06:25.053617 11669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:25.040420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:53353 (local address 127.25.130.62:43125)
0812 19:06:25.040687 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:25.040691 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:25.041583 (+ 892us) server_negotiation.cc:407] Connection header received
0812 19:06:25.042959 (+ 1376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:25.042962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:25.043015 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:25.043096 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:25.044932 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:25.045669 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:25.047064 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:25.047383 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:25.047479 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:25.050111 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:25.050114 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:25.050116 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:25.050137 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:25.051483 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:25.051989 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:25.051993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:25.051995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:25.052108 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:25.052448 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:25.052451 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:25.052453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:25.052649 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:25.052739 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:25.053262 (+ 523us) server_negotiation.cc:299] Negotiation successful
0812 19:06:25.053403 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":207,"thread_start_us":100,"threads_started":1}
I20260812 19:06:25.054458 11670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:25.040933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.3:53353)
0812 19:06:25.041413 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:25.041486 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:25.042781 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:25.043633 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:25.043642 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:25.044126 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:06:25.044788 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:25.044806 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:25.045805 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:25.045808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:25.046961 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:25.046972 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:25.047551 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:25.047556 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:25.047784 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:25.048359 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:06:25.048377 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:06:25.049964 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:06:25.051587 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:25.051591 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:25.051604 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:25.051870 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:25.052192 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:25.052195 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:25.052197 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:25.052357 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:25.052751 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:06:25.052759 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:06:25.053019 (+ 260us) client_negotiation.cc:769] Sending connection context
0812 19:06:25.053203 (+ 184us) client_negotiation.cc:240] Negotiation successful
0812 19:06:25.053593 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":117,"threads_started":1}
I20260812 19:06:25.055588 11666 heartbeater.cc:347] Connected to a master server at 127.25.130.62:43125
I20260812 19:06:25.055845 11666 heartbeater.cc:464] Registering TS with master...
I20260812 19:06:25.056366 11666 heartbeater.cc:511] Master 127.25.130.62:43125 requested a full tablet report, sending...
I20260812 19:06:25.057368 11095 ts_manager.cc:194] Registered new tserver with Master: 16ebfcb8b3dc4ab885bb6138969fc8f1 (127.25.130.3:46001)
I20260812 19:06:25.057968 11095 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.3@KRBTEST.COM'} at 127.25.130.3:53353
I20260812 19:06:25.061280 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:06:25.065419 11669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:25.061894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43388 (local address 127.25.130.62:43125)
0812 19:06:25.062012 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:25.062015 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:25.062227 (+ 212us) server_negotiation.cc:407] Connection header received
0812 19:06:25.062378 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:25.062380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:25.062480 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:25.062561 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:06:25.062749 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:25.063263 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:25.064146 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:25.064365 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:25.064428 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:25.064713 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:06:25.065101 (+ 388us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:06:25.065198 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:25.065227 (+ 29us) server_negotiation.cc:299] Negotiation successful
0812 19:06:25.065281 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 19:06:25.066115 11095 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:43388:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:06:25.069344 11095 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:06:25.077558 11239 tablet_service.cc:1467] Processing CreateTablet for tablet f78b104c96db4a72839fc74e982a2ee7 (DEFAULT_TABLE table=test-table [id=700b0c13b79e4aa7a10ef12d2127900a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:06:25.078307 11239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f78b104c96db4a72839fc74e982a2ee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:25.081291 11191 tablet_bootstrap.cc:492] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c: Bootstrap starting.
I20260812 19:06:25.082285 11191 tablet_bootstrap.cc:654] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:25.082391 11326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:25.076357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:46001 (local address 127.0.0.1:55280)
0812 19:06:25.076471 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:25.076488 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:25.076571 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:25.077686 (+ 1115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:25.077689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:25.077708 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:06:25.077829 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:25.077836 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:25.079731 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:25.079735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:25.080567 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:25.080578 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:25.081908 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:25.081912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:25.082013 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:25.082069 (+ 56us) client_negotiation.cc:769] Sending connection context
0812 19:06:25.082145 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 19:06:25.082237 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":49}
I20260812 19:06:25.082650 11680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:25.076886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53636 (local address 127.25.130.2:41435)
0812 19:06:25.077325 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:25.077336 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:25.077360 (+ 24us) server_negotiation.cc:407] Connection header received
0812 19:06:25.077432 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:25.077437 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:25.077653 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:25.077806 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:06:25.078085 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:25.079244 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:25.080765 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:25.081926 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:25.082028 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:25.082184 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:25.082378 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 19:06:25.082494 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":123,"threads_started":1}
I20260812 19:06:25.082785 11676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:25.076620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:41435 (local address 127.0.0.1:53636)
0812 19:06:25.076937 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:25.076952 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:25.077036 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:25.077866 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:25.077869 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:25.077884 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:06:25.077962 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:25.077967 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:25.079387 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:25.079390 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:25.080611 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:25.080623 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:25.082154 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:25.082157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:25.082238 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:25.082258 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 19:06:25.082324 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 19:06:25.082386 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":1,"thread_start_us":126,"threads_started":1}
I20260812 19:06:25.083410 11679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:25.076701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55280 (local address 127.25.130.3:46001)
0812 19:06:25.077214 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:25.077224 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:25.077245 (+ 21us) server_negotiation.cc:407] Connection header received
0812 19:06:25.077312 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:25.077317 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:25.077539 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:25.077697 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:06:25.078142 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:25.079598 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:25.080701 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:25.081794 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:25.081896 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:25.082067 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:25.083057 (+ 990us) server_negotiation.cc:299] Negotiation successful
0812 19:06:25.083261 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":120,"threads_started":1}
I20260812 19:06:25.084045 11191 tablet_bootstrap.cc:492] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c: No bootstrap required, opened a new log
I20260812 19:06:25.084151 11191 ts_tablet_manager.cc:1397] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 19:06:25.084678 11191 raft_consensus.cc:348] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } }
I20260812 19:06:25.084955 11191 raft_consensus.cc:374] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:25.085057 11191 raft_consensus.cc:729] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7402118b06af4bb1bc693820faf92a6c, State: Initialized, Role: FOLLOWER
I20260812 19:06:25.085301 11191 consensus_queue.cc:260] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } }
I20260812 19:06:25.085696 11191 ts_tablet_manager.cc:1428] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260812 19:06:25.086243 11587 tablet_service.cc:1467] Processing CreateTablet for tablet f78b104c96db4a72839fc74e982a2ee7 (DEFAULT_TABLE table=test-table [id=700b0c13b79e4aa7a10ef12d2127900a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:06:25.086243 11407 tablet_service.cc:1467] Processing CreateTablet for tablet f78b104c96db4a72839fc74e982a2ee7 (DEFAULT_TABLE table=test-table [id=700b0c13b79e4aa7a10ef12d2127900a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:06:25.087466 11407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f78b104c96db4a72839fc74e982a2ee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:25.087474 11587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f78b104c96db4a72839fc74e982a2ee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:25.096280 11683 tablet_bootstrap.cc:492] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1: Bootstrap starting.
I20260812 19:06:25.097776 11682 tablet_bootstrap.cc:492] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4: Bootstrap starting.
I20260812 19:06:25.098824 11683 tablet_bootstrap.cc:654] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:25.099733 11683 log.cc:824] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1: Log is configured to *not* fsync() on all Append() calls
I20260812 19:06:25.100080 11682 tablet_bootstrap.cc:654] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:25.100750 11682 log.cc:824] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4: Log is configured to *not* fsync() on all Append() calls
I20260812 19:06:25.101825 11683 tablet_bootstrap.cc:492] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1: No bootstrap required, opened a new log
I20260812 19:06:25.102012 11683 ts_tablet_manager.cc:1397] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 19:06:25.102581 11682 tablet_bootstrap.cc:492] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4: No bootstrap required, opened a new log
I20260812 19:06:25.102751 11682 ts_tablet_manager.cc:1397] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 19:06:25.105111 11683 raft_consensus.cc:348] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } }
I20260812 19:06:25.105352 11683 raft_consensus.cc:374] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:25.105387 11683 raft_consensus.cc:729] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16ebfcb8b3dc4ab885bb6138969fc8f1, State: Initialized, Role: FOLLOWER
I20260812 19:06:25.105697 11682 raft_consensus.cc:348] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } }
I20260812 19:06:25.105877 11683 consensus_queue.cc:260] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } }
I20260812 19:06:25.105958 11682 raft_consensus.cc:374] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:25.106074 11682 raft_consensus.cc:729] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 122a6c94061e4dc394286c9ca63162c4, State: Initialized, Role: FOLLOWER
I20260812 19:06:25.106539 11682 consensus_queue.cc:260] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } }
I20260812 19:06:25.106765 11666 heartbeater.cc:503] Master 127.25.130.62:43125 was elected leader, sending a full tablet report...
I20260812 19:06:25.107100 11683 ts_tablet_manager.cc:1428] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 19:06:25.107312 11504 heartbeater.cc:503] Master 127.25.130.62:43125 was elected leader, sending a full tablet report...
I20260812 19:06:25.108615 11682 ts_tablet_manager.cc:1428] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260812 19:06:25.158973 11318 tablet.cc:2369] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:06:25.168419 11687 raft_consensus.cc:482] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:25.168653 11687 raft_consensus.cc:504] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } }
I20260812 19:06:25.169839 11687 leader_election.cc:288] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 122a6c94061e4dc394286c9ca63162c4 (127.25.130.2:41435), 7402118b06af4bb1bc693820faf92a6c (127.25.130.1:32921)
I20260812 19:06:25.173892 11680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:25.170241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:55531 (local address 127.25.130.2:41435)
0812 19:06:25.170352 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:25.170356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:25.170680 (+ 324us) server_negotiation.cc:407] Connection header received
0812 19:06:25.170802 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:25.170805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:25.170858 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:25.170936 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:06:25.171208 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:25.171689 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:25.172833 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:25.173331 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:25.173468 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:25.173528 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:25.173690 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 19:06:25.173761 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 19:06:25.173933 11689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:25.170101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:41435 (local address 127.25.130.3:55531)
0812 19:06:25.170562 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:25.170575 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:25.170703 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:25.170957 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:25.170959 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:25.170973 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:06:25.171097 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:25.171105 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:25.171798 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:25.171801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:25.172706 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:25.172716 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:25.173468 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:25.173472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:25.173574 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:25.173596 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 19:06:25.173662 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 19:06:25.173746 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":80,"thread_start_us":115,"threads_started":1}
I20260812 19:06:25.174407 11327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:25.170209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:57843 (local address 127.25.130.1:32921)
0812 19:06:25.170345 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:25.170350 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:25.170410 (+ 60us) server_negotiation.cc:407] Connection header received
0812 19:06:25.170565 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:25.170569 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:25.170645 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:25.170752 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:06:25.171064 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:25.171665 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:25.172847 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:25.173333 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:25.173969 (+ 636us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:25.174154 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:25.174205 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 19:06:25.174287 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 19:06:25.173941 11670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:25.170101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:32921 (local address 127.25.130.3:57843)
0812 19:06:25.170320 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:25.170336 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:25.170453 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:25.170778 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:25.170780 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:25.170798 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:06:25.170935 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:25.170942 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:25.171788 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:25.171792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:25.172707 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:25.172717 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:25.173524 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:25.173528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:25.173641 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:25.173666 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 19:06:25.173727 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 19:06:25.173797 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":90}
I20260812 19:06:25.174823 11435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f78b104c96db4a72839fc74e982a2ee7" candidate_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "122a6c94061e4dc394286c9ca63162c4" is_pre_election: true
I20260812 19:06:25.175066 11261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f78b104c96db4a72839fc74e982a2ee7" candidate_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7402118b06af4bb1bc693820faf92a6c" is_pre_election: true
I20260812 19:06:25.175134 11435 raft_consensus.cc:2454] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16ebfcb8b3dc4ab885bb6138969fc8f1 in term 0.
I20260812 19:06:25.175329 11261 raft_consensus.cc:2454] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16ebfcb8b3dc4ab885bb6138969fc8f1 in term 0.
I20260812 19:06:25.175601 11530 leader_election.cc:302] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 122a6c94061e4dc394286c9ca63162c4, 16ebfcb8b3dc4ab885bb6138969fc8f1; no voters:
I20260812 19:06:25.175874 11687 raft_consensus.cc:2781] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:25.175966 11687 raft_consensus.cc:482] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:25.176016 11687 raft_consensus.cc:3037] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:25.177053 11687 raft_consensus.cc:504] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } }
I20260812 19:06:25.177419 11687 leader_election.cc:288] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [CANDIDATE]: Term 1 election: Requested vote from peers 122a6c94061e4dc394286c9ca63162c4 (127.25.130.2:41435), 7402118b06af4bb1bc693820faf92a6c (127.25.130.1:32921)
I20260812 19:06:25.177819 11261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f78b104c96db4a72839fc74e982a2ee7" candidate_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7402118b06af4bb1bc693820faf92a6c"
I20260812 19:06:25.177932 11261 raft_consensus.cc:3037] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:25.178944 11261 raft_consensus.cc:2454] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16ebfcb8b3dc4ab885bb6138969fc8f1 in term 1.
I20260812 19:06:25.179183 11435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f78b104c96db4a72839fc74e982a2ee7" candidate_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "122a6c94061e4dc394286c9ca63162c4"
I20260812 19:06:25.179234 11528 leader_election.cc:302] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16ebfcb8b3dc4ab885bb6138969fc8f1, 7402118b06af4bb1bc693820faf92a6c; no voters:
I20260812 19:06:25.179304 11435 raft_consensus.cc:3037] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:25.179402 11687 raft_consensus.cc:2781] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:25.179615 11687 raft_consensus.cc:686] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [term 1 LEADER]: Becoming Leader. State: Replica: 16ebfcb8b3dc4ab885bb6138969fc8f1, State: Running, Role: LEADER
I20260812 19:06:25.179953 11687 consensus_queue.cc:237] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } }
I20260812 19:06:25.180459 11435 raft_consensus.cc:2454] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16ebfcb8b3dc4ab885bb6138969fc8f1 in term 1.
I20260812 19:06:25.183185 11095 catalog_manager.cc:5495] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16ebfcb8b3dc4ab885bb6138969fc8f1 (127.25.130.3). New cstate: current_term: 1 leader_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:25.206715 11679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:25.203019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55296 (local address 127.25.130.3:46001)
0812 19:06:25.203150 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:25.203153 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:25.203166 (+ 13us) server_negotiation.cc:407] Connection header received
0812 19:06:25.203247 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:25.203249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:25.203307 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:25.203397 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:06:25.203604 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:25.204197 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:25.205024 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:25.205278 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:25.205370 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:25.205763 (+ 393us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:06:25.206226 (+ 463us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:06:25.206431 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:25.206467 (+ 36us) server_negotiation.cc:299] Negotiation successful
0812 19:06:25.206534 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 19:06:25.212855 11261 raft_consensus.cc:1261] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [term 1 FOLLOWER]: Refusing update from remote peer 16ebfcb8b3dc4ab885bb6138969fc8f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:25.212858 11435 raft_consensus.cc:1261] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4 [term 1 FOLLOWER]: Refusing update from remote peer 16ebfcb8b3dc4ab885bb6138969fc8f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:25.213549 11690 consensus_queue.cc:1035] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:25.213737 11687 consensus_queue.cc:1035] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:25.221781 11694 mvcc.cc:205] Tried to move back new op lower bound from 7317756253024681984 to 7317756252899840000. Current Snapshot: MvccSnapshot[applied={T|T < 7317756253024681984 or (T in {7317756253024681984})}]
I20260812 19:06:25.223956 11697 mvcc.cc:205] Tried to move back new op lower bound from 7317756253024681984 to 7317756252899840000. Current Snapshot: MvccSnapshot[applied={T|T < 7317756253024681984 or (T in {7317756253024681984})}]
I20260812 19:06:25.227818 11095 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:43388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:06:25.227993 11095 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:43388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:06:25.230471 11095 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7: Sending DeleteTablet for 3 replicas of tablet f78b104c96db4a72839fc74e982a2ee7
I20260812 19:06:25.231138 11587 tablet_service.cc:1514] Processing DeleteTablet for tablet f78b104c96db4a72839fc74e982a2ee7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:06:25 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:55280
I20260812 19:06:25.231343 11239 tablet_service.cc:1514] Processing DeleteTablet for tablet f78b104c96db4a72839fc74e982a2ee7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:06:25 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:56742
I20260812 19:06:25.231657 11407 tablet_service.cc:1514] Processing DeleteTablet for tablet f78b104c96db4a72839fc74e982a2ee7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:06:25 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:53636
I20260812 19:06:25.231755 11708 tablet_replica.cc:330] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c: stopping tablet replica
I20260812 19:06:25.231873 11708 raft_consensus.cc:2229] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:25.232025 11708 raft_consensus.cc:2258] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:25.232314 11707 tablet_replica.cc:330] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1: stopping tablet replica
I20260812 19:06:25.232323 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 11137
I20260812 19:06:25.232610 11707 raft_consensus.cc:2229] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:06:25.232910 11707 raft_consensus.cc:2258] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:25.233218 11709 tablet_replica.cc:330] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4: stopping tablet replica
I20260812 19:06:25.233475 11709 raft_consensus.cc:2229] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:25.233716 11709 raft_consensus.cc:2258] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:25.235697 11707 ts_tablet_manager.cc:1905] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:06:25.237995 11707 ts_tablet_manager.cc:1918] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:06:25.238111 11707 log.cc:1196] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/wal/wals/f78b104c96db4a72839fc74e982a2ee7
I20260812 19:06:25.238504 11707 ts_tablet_manager.cc:1939] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1: Deleting consensus metadata
I20260812 19:06:25.239714 11082 catalog_manager.cc:4841] TS 16ebfcb8b3dc4ab885bb6138969fc8f1 (127.25.130.3:46001): tablet f78b104c96db4a72839fc74e982a2ee7 (table test-table [id=700b0c13b79e4aa7a10ef12d2127900a]) successfully deleted
I20260812 19:06:25.246773 11709 ts_tablet_manager.cc:1905] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 19:06:25.247104 11079 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.130.1:32921 (error 108)
I20260812 19:06:25.247308 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 11321
I20260812 19:06:25.250768 11676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:25.250472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:32921 (local address 127.0.0.1:56746)
0812 19:06:25.250616 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:25.250688 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.1:32921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 19:06:25.250932 11079 catalog_manager.cc:4584] TS 7402118b06af4bb1bc693820faf92a6c (127.25.130.1:32921): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f78b104c96db4a72839fc74e982a2ee7: Network error: Client connection negotiation failed: client connection to 127.25.130.1:32921: connect: Connection refused (error 111)
I20260812 19:06:25.257633 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 11506
I20260812 19:06:25.257855 11676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:25.257387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:41435 (local address 127.0.0.1:53648)
0812 19:06:25.257697 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:25.257757 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.2:41435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 19:06:25.266498 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 11053
I20260812 19:06:25.275944 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:43125
--webserver_interface=127.25.130.62
--webserver_port=46101
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:35319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:43125
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:06:25.389516 11712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:06:25.389811 11712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:06:25.389853 11712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:06:25.393949 11712 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:06:25.394019 11712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:06:25.394037 11712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:06:25.394064 11712 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:06:25.394099 11712 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:06:25.399621 11712 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:35319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:43125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:43125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=46101
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:06:25.400799 11712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:06:25.401903 11712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:25.410596 11717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:25.410580 11719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:25.410684 11712 server_base.cc:1034] running on GCE node
W20260812 19:06:25.410580 11722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:25.411453 11712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:06:25.412546 11712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:06:25.413699 11712 hybrid_clock.cc:648] HybridClock initialized: now 1786561585413684 us; error 31 us; skew 500 ppm
Aug 12 19:06:25 dist-test-slave-49h3 krb5kdc[10388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561585, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:25.416285 11712 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:06:25.417176 11712 webserver.cc:460] Webserver started at http://127.25.130.62:46101/ using document root <none> and password file <none>
I20260812 19:06:25.417752 11712 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:25.417821 11712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:25.421633 11712 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.003s
I20260812 19:06:25.423710 11730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:25.424932 11712 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:06:25.425055 11712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "e6b8bf975275492ab331eb132c0d92b7"
format_stamp: "Formatted at 2026-08-12 19:06:23 on dist-test-slave-49h3"
server_key: "e871a9b3f2cd887aeb459bca1de2a578"
server_key_iv: "f64f5fb3e4733f5f5f1b25e46c08aa5a"
server_key_version: "encryptionkey@0"
I20260812 19:06:25.425681 11712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:25.438980 11712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:25.441833 11712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:06:25.442013 11712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:25.449074 11712 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:43125
I20260812 19:06:25.449355 11797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:43125 every 8 connection(s)
I20260812 19:06:25.450222 11712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:06:25.452904 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 11712
I20260812 19:06:25.453404 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:32921
--local_ip_for_outbound_sockets=127.25.130.1
--tserver_master_addrs=127.25.130.62:43125
--webserver_port=38213
--webserver_interface=127.25.130.1
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:35319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:06:25.455214 11798 sys_catalog.cc:263] Verifying existing consensus state
I20260812 19:06:25.456990 11798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7: Bootstrap starting.
I20260812 19:06:25.475404 11798 log.cc:824] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7: Log is configured to *not* fsync() on all Append() calls
I20260812 19:06:25.486989 11798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:06:25.487396 11798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7: Bootstrap complete.
I20260812 19:06:25.491565 11798 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b8bf975275492ab331eb132c0d92b7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 43125 } }
I20260812 19:06:25.491984 11798 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b8bf975275492ab331eb132c0d92b7, State: Initialized, Role: FOLLOWER
I20260812 19:06:25.492462 11798 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b8bf975275492ab331eb132c0d92b7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 43125 } }
I20260812 19:06:25.492580 11798 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:06:25.492642 11798 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:06:25.492743 11798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:06:25.494844 11798 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b8bf975275492ab331eb132c0d92b7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 43125 } }
I20260812 19:06:25.495174 11798 leader_election.cc:302] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6b8bf975275492ab331eb132c0d92b7; no voters:
I20260812 19:06:25.495488 11798 leader_election.cc:288] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 19:06:25.495623 11803 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:06:25.495811 11803 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [term 3 LEADER]: Becoming Leader. State: Replica: e6b8bf975275492ab331eb132c0d92b7, State: Running, Role: LEADER
I20260812 19:06:25.496181 11803 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b8bf975275492ab331eb132c0d92b7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 43125 } }
I20260812 19:06:25.496827 11798 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:06:25.497684 11804 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e6b8bf975275492ab331eb132c0d92b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b8bf975275492ab331eb132c0d92b7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 43125 } } }
I20260812 19:06:25.497840 11804 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [sys.catalog]: This master's current role is: LEADER
I20260812 19:06:25.498243 11805 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6b8bf975275492ab331eb132c0d92b7. Latest consensus state: current_term: 3 leader_uuid: "e6b8bf975275492ab331eb132c0d92b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b8bf975275492ab331eb132c0d92b7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 43125 } } }
I20260812 19:06:25.498387 11805 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6b8bf975275492ab331eb132c0d92b7 [sys.catalog]: This master's current role is: LEADER
I20260812 19:06:25.498679 11814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:06:25.503950 11814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:06:25.504617 11814 catalog_manager.cc:1259] Loaded cluster ID: dab9e87dc88d41f4b59fc9335882d6f0
I20260812 19:06:25.504721 11814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 19:06:25.505939 11814 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f338cdaf9ad google::LogMessage::Fail() at ??:0
@ 0x7f338cdb288a google::LogMessage::SendToLog() at ??:0
@ 0x7f338cdaf4de google::LogMessage::Flush() at ??:0
@ 0x7f338cdb0a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f33976eefdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f33976efd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f33976ec579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f3397730640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f338f0013d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f338f001c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f338f0033f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f338eff2512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f33906366db start_thread at ??:0
@ 0x7f338b8ad61f clone at ??:0
W20260812 19:06:25.584736 11801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:06:25.584987 11801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:06:25.585031 11801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:06:25.589269 11801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:06:25.589344 11801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:06:25.589422 11801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:06:25.595191 11801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:35319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:32921
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=38213
--tserver_master_addrs=127.25.130.62:43125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:06:25.596390 11801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:06:25.597514 11801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:25.607679 11847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:25.607851 11801 server_base.cc:1034] running on GCE node
W20260812 19:06:25.607889 11850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:25.607677 11848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:25.608403 11801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:06:25.609082 11801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:06:25.610270 11801 hybrid_clock.cc:648] HybridClock initialized: now 1786561585610251 us; error 29 us; skew 500 ppm
Aug 12 19:06:25 dist-test-slave-49h3 krb5kdc[10388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561585, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:25.613032 11801 init.cc:377] Logged in from keytab as kudu/127.25.130.1@KRBTEST.COM (short username kudu)
I20260812 19:06:25.614068 11801 webserver.cc:460] Webserver started at http://127.25.130.1:38213/ using document root <none> and password file <none>
I20260812 19:06:25.614645 11801 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:25.614710 11801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:25.619073 11801 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 19:06:25.621573 11859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:25.623008 11801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:06:25.623136 11801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "7402118b06af4bb1bc693820faf92a6c"
format_stamp: "Formatted at 2026-08-12 19:06:23 on dist-test-slave-49h3"
server_key: "ee20f51d119e3aa2ff4faaa780658c08"
server_key_iv: "a667981a3b7f9c6654d32065cc6e2c77"
server_key_version: "encryptionkey@0"
I20260812 19:06:25.623730 11801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:25.650348 11801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:25.653878 11801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:06:25.654078 11801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:25.654739 11801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:06:25.655900 11868 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 19:06:25.661659 11801 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 19:06:25.661738 11801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260812 19:06:25.661809 11801 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 19:06:25.663606 11801 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 19:06:25.663666 11801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260812 19:06:25.664034 11868 tablet_bootstrap.cc:492] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c: Bootstrap starting.
I20260812 19:06:25.674044 11801 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:32921
I20260812 19:06:25.674239 12000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:32921 every 8 connection(s)
I20260812 19:06:25.675182 11801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:06:25.681502 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 11801
I20260812 19:06:25.681936 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:41435
--local_ip_for_outbound_sockets=127.25.130.2
--tserver_master_addrs=127.25.130.62:43125
--webserver_port=38827
--webserver_interface=127.25.130.2
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:35319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:06:25.688987 11868 log.cc:824] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c: Log is configured to *not* fsync() on all Append() calls
I20260812 19:06:25.696429 11868 tablet_bootstrap.cc:492] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:06:25.696803 11868 tablet_bootstrap.cc:492] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c: Bootstrap complete.
I20260812 19:06:25.697928 11868 ts_tablet_manager.cc:1397] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c: Time spent bootstrapping tablet: real 0.034s user 0.023s sys 0.007s
I20260812 19:06:25.700352 11868 raft_consensus.cc:348] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } }
I20260812 19:06:25.700773 11868 raft_consensus.cc:729] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7402118b06af4bb1bc693820faf92a6c, State: Initialized, Role: FOLLOWER
I20260812 19:06:25.701263 11868 consensus_queue.cc:260] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } }
I20260812 19:06:25.702169 11868 ts_tablet_manager.cc:1428] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
W20260812 19:06:25.764389 12003 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:06:25.677215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.1:55283)
0812 19:06:25.677773 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:25.677829 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:25.678671 (+ 842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:25.763571 (+ 84900us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":153,"threads_started":1}
W20260812 19:06:25.764989 12001 heartbeater.cc:644] Failed to heartbeat to 127.25.130.62:43125 (0 consecutive failures): Network error: Failed to ping master at 127.25.130.62:43125: Client connection negotiation failed: client connection to 127.25.130.62:43125: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 19:06:25.765770 12003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:25.765504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.1:49145)
0812 19:06:25.765628 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:25.765701 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 19:06:25.766409 12003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:25.766222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.1:48091)
0812 19:06:25.766315 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:25.766356 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 19:06:25.766573 12001 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 19:06:25.824123 12004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:06:25.824404 12004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:06:25.824447 12004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:06:25.828586 12004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:06:25.828656 12004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:06:25.828717 12004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:06:25.834055 12004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:35319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:41435
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=38827
--tserver_master_addrs=127.25.130.62:43125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:06:25.835088 12004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:06:25.836057 12004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:25.845793 12012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:25.845784 12011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:25.845829 12004 server_base.cc:1034] running on GCE node
W20260812 19:06:25.845784 12014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:25.846551 12004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:06:25.847205 12004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:06:25.848367 12004 hybrid_clock.cc:648] HybridClock initialized: now 1786561585848347 us; error 32 us; skew 500 ppm
Aug 12 19:06:25 dist-test-slave-49h3 krb5kdc[10388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561585, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:25.851183 12004 init.cc:377] Logged in from keytab as kudu/127.25.130.2@KRBTEST.COM (short username kudu)
I20260812 19:06:25.852182 12004 webserver.cc:460] Webserver started at http://127.25.130.2:38827/ using document root <none> and password file <none>
I20260812 19:06:25.852759 12004 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:25.852834 12004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:25.856851 12004 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 19:06:25.859195 12022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:25.860509 12004 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 19:06:25.860627 12004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "122a6c94061e4dc394286c9ca63162c4"
format_stamp: "Formatted at 2026-08-12 19:06:23 on dist-test-slave-49h3"
server_key: "88dc95548b82ec0854df976f5f3f910d"
server_key_iv: "d8a997efc64fad3f03f82471bfd77298"
server_key_version: "encryptionkey@0"
I20260812 19:06:25.861191 12004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:25.877027 12004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:25.880105 12004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:06:25.880322 12004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:25.880990 12004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:06:25.882290 12029 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 19:06:25.886904 12004 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 19:06:25.886992 12004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260812 19:06:25.887058 12004 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 19:06:25.888860 12004 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 19:06:25.888918 12004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260812 19:06:25.889189 12029 tablet_bootstrap.cc:492] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4: Bootstrap starting.
I20260812 19:06:25.895262 12029 log.cc:824] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4: Log is configured to *not* fsync() on all Append() calls
I20260812 19:06:25.899022 12004 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:41435
I20260812 19:06:25.899092 12156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:41435 every 8 connection(s)
I20260812 19:06:25.899911 12004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:06:25.900904 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 12004
I20260812 19:06:25.901341 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:46001
--local_ip_for_outbound_sockets=127.25.130.3
--tserver_master_addrs=127.25.130.62:43125
--webserver_port=45187
--webserver_interface=127.25.130.3
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:35319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:06:25.903560 12163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:25.902034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.2:56183)
0812 19:06:25.902576 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:25.902714 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":124,"threads_started":1}
W20260812 19:06:25.903957 12159 heartbeater.cc:644] Failed to heartbeat to 127.25.130.62:43125 (0 consecutive failures): Network error: Failed to ping master at 127.25.130.62:43125: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
I20260812 19:06:25.904083 12029 tablet_bootstrap.cc:492] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:06:25.904532 12029 tablet_bootstrap.cc:492] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4: Bootstrap complete.
I20260812 19:06:25.904776 12165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:25.904357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.2:54103)
0812 19:06:25.904621 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:25.904679 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":72,"threads_started":1}
I20260812 19:06:25.905509 12029 ts_tablet_manager.cc:1397] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4: Time spent bootstrapping tablet: real 0.017s user 0.016s sys 0.000s
I20260812 19:06:25.906165 12163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:25.905928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.2:54073)
0812 19:06:25.906048 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:25.906103 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 19:06:25.906344 12159 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 19:06:25.908605 12029 raft_consensus.cc:348] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } }
I20260812 19:06:25.909092 12029 raft_consensus.cc:729] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 122a6c94061e4dc394286c9ca63162c4, State: Initialized, Role: FOLLOWER
I20260812 19:06:25.909608 12029 consensus_queue.cc:260] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } }
I20260812 19:06:25.910589 12029 ts_tablet_manager.cc:1428] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260812 19:06:25.911147 12163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:25.910646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.2:32807)
0812 19:06:25.910799 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:25.910858 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 19:06:25.911783 12163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:25.911600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.2:39057)
0812 19:06:25.911690 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:25.911733 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 19:06:26.020989 12164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:06:26.021302 12164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:06:26.021495 12164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:06:26.025705 12164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:06:26.025780 12164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:06:26.025841 12164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:06:26.031474 12164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:35319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:46001
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=45187
--tserver_master_addrs=127.25.130.62:43125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:06:26.032568 12164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:06:26.033615 12164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:26.043318 12172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:26.043320 12171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:26.043321 12174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:26.043653 12164 server_base.cc:1034] running on GCE node
I20260812 19:06:26.044189 12164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:06:26.044888 12164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:06:26.046082 12164 hybrid_clock.cc:648] HybridClock initialized: now 1786561586046043 us; error 34 us; skew 500 ppm
Aug 12 19:06:26 dist-test-slave-49h3 krb5kdc[10388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561586, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:26.048884 12164 init.cc:377] Logged in from keytab as kudu/127.25.130.3@KRBTEST.COM (short username kudu)
I20260812 19:06:26.049881 12164 webserver.cc:460] Webserver started at http://127.25.130.3:45187/ using document root <none> and password file <none>
I20260812 19:06:26.050460 12164 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:26.050535 12164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:26.054777 12164 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260812 19:06:26.057045 12184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:26.058418 12164 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 19:06:26.058547 12164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1"
format_stamp: "Formatted at 2026-08-12 19:06:23 on dist-test-slave-49h3"
server_key: "816763db7740ab739d34f87b6d5f4ee8"
server_key_iv: "b40bf501143fff9ea299f2e64312d603"
server_key_version: "encryptionkey@0"
I20260812 19:06:26.059150 12164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:26.077168 12164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:26.080370 12164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:06:26.080578 12164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:26.081243 12164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:06:26.082217 12164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:06:26.082271 12164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:26.082329 12164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:06:26.082360 12164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:26.091526 12164 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:46001
I20260812 19:06:26.091611 12318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:46001 every 8 connection(s)
I20260812 19:06:26.092623 12164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:06:26.096760 12321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:26.094928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.3:36077)
0812 19:06:26.095453 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:26.095885 (+ 432us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":131,"threads_started":1}
W20260812 19:06:26.097245 12319 heartbeater.cc:644] Failed to heartbeat to 127.25.130.62:43125 (0 consecutive failures): Network error: Failed to ping master at 127.25.130.62:43125: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
I20260812 19:06:26.097944 12321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:26.097706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.3:48707)
0812 19:06:26.097828 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:26.097883 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 19:06:26.098541 12321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:26.098339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.3:40391)
0812 19:06:26.098443 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:26.098490 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 19:06:26.098716 12319 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 19:06:26.099982 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 12164
I20260812 19:06:26.767700 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:26.767208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.1:51967)
0812 19:06:26.767506 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:26.767572 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":88,"threads_started":1}
I20260812 19:06:26.913237 12325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:26.912657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.2:43923)
0812 19:06:26.913032 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:26.913107 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":148,"threads_started":1}
I20260812 19:06:26.992744 12326 raft_consensus.cc:482] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:26.993042 12326 raft_consensus.cc:504] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } }
I20260812 19:06:26.994609 12326 leader_election.cc:288] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 16ebfcb8b3dc4ab885bb6138969fc8f1 (127.25.130.3:46001), 122a6c94061e4dc394286c9ca63162c4 (127.25.130.2:41435)
Aug 12 19:06:27 dist-test-slave-49h3 krb5kdc[10388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561585, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for kudu/127.25.130.2@KRBTEST.COM
Aug 12 19:06:27 dist-test-slave-49h3 krb5kdc[10388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561585, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for kudu/127.25.130.3@KRBTEST.COM
I20260812 19:06:27.010227 12328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:26.994999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:46001 (local address 127.25.130.1:35897)
0812 19:06:26.995406 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:26.995423 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:26.995547 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:26.997597 (+ 2050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:26.997599 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:26.997835 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:06:26.998427 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:26.998445 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:26.999847 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:26.999849 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:27.000788 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:27.000799 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:27.001600 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:27.001604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:27.001830 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:27.002612 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:06:27.002624 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:06:27.004626 (+ 2002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:06:27.007313 (+ 2687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:27.007316 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:27.007318 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:27.007754 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:27.008433 (+ 679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:27.008435 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:27.008437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:27.008504 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:27.009139 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:06:27.009141 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:06:27.009318 (+ 177us) client_negotiation.cc:769] Sending connection context
0812 19:06:27.009540 (+ 222us) client_negotiation.cc:240] Negotiation successful
0812 19:06:27.009996 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":206,"spinlock_wait_cycles":10880,"thread_start_us":94,"threads_started":1}
I20260812 19:06:27.010254 12331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:26.995238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:45103 (local address 127.25.130.2:41435)
0812 19:06:26.995697 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:26.995718 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:26.995777 (+ 59us) server_negotiation.cc:407] Connection header received
0812 19:06:26.996105 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:26.996111 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:26.996632 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:26.996962 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:26.998660 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:26.999678 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:27.000946 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:27.001486 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:27.001652 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:27.004540 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:27.004548 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:27.004563 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:27.004603 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:27.007128 (+ 2525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:27.007892 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:27.007899 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:27.007905 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:27.008069 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:27.008410 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:27.008412 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:27.008414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:27.008925 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:27.009034 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:27.009699 (+ 665us) server_negotiation.cc:299] Negotiation successful
0812 19:06:27.010010 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":121,"threads_started":1}
I20260812 19:06:27.010236 12333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:26.995248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:35897 (local address 127.25.130.3:46001)
0812 19:06:26.995775 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:26.995787 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:26.995824 (+ 37us) server_negotiation.cc:407] Connection header received
0812 19:06:26.996779 (+ 955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:26.996788 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:26.997223 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:26.997612 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:26.998626 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:26.999691 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:27.000929 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:27.001436 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:27.001648 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:27.004852 (+ 3204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:27.004859 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:27.004874 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:27.004914 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:27.007128 (+ 2214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:27.007906 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:27.007913 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:27.007919 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:27.008069 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:27.008591 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:27.008595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:27.008597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:27.009022 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:27.009155 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:27.009706 (+ 551us) server_negotiation.cc:299] Negotiation successful
0812 19:06:27.009999 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":90,"threads_started":1}
I20260812 19:06:27.010279 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:26.994968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:41435 (local address 127.25.130.1:45103)
0812 19:06:26.995211 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:26.995228 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:26.995380 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:26.997074 (+ 1694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:26.997080 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:26.997828 (+ 748us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:06:26.998428 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:26.998445 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:26.999851 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:26.999854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:27.000789 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:27.000800 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:27.001627 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:27.001631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:27.001834 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:27.002574 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:06:27.002598 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:06:27.004331 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:06:27.007258 (+ 2927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:27.007265 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:27.007282 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:27.007731 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:27.008173 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:27.008176 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:27.008179 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:27.008330 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:27.009046 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:06:27.009053 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:06:27.009320 (+ 267us) client_negotiation.cc:769] Sending connection context
0812 19:06:27.009557 (+ 237us) client_negotiation.cc:240] Negotiation successful
0812 19:06:27.010084 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":272,"spinlock_wait_cycles":62208}
I20260812 19:06:27.012109 12105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f78b104c96db4a72839fc74e982a2ee7" candidate_uuid: "7402118b06af4bb1bc693820faf92a6c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "122a6c94061e4dc394286c9ca63162c4" is_pre_election: true
I20260812 19:06:27.012522 12105 raft_consensus.cc:2454] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7402118b06af4bb1bc693820faf92a6c in term 1.
I20260812 19:06:27.013419 11861 leader_election.cc:302] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 122a6c94061e4dc394286c9ca63162c4, 7402118b06af4bb1bc693820faf92a6c; no voters:
W20260812 19:06:27.013677 11863 leader_election.cc:341] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 16ebfcb8b3dc4ab885bb6138969fc8f1 (127.25.130.3:46001): Not found: Tablet not found: f78b104c96db4a72839fc74e982a2ee7
I20260812 19:06:27.013782 12326 raft_consensus.cc:2781] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 19:06:27.013892 12326 raft_consensus.cc:482] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:27.013947 12326 raft_consensus.cc:3037] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:27.016520 12326 raft_consensus.cc:504] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } }
I20260812 19:06:27.016979 12326 leader_election.cc:288] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [CANDIDATE]: Term 2 election: Requested vote from peers 16ebfcb8b3dc4ab885bb6138969fc8f1 (127.25.130.3:46001), 122a6c94061e4dc394286c9ca63162c4 (127.25.130.2:41435)
I20260812 19:06:27.017616 12105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f78b104c96db4a72839fc74e982a2ee7" candidate_uuid: "7402118b06af4bb1bc693820faf92a6c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "122a6c94061e4dc394286c9ca63162c4"
W20260812 19:06:27.017794 11863 leader_election.cc:341] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 16ebfcb8b3dc4ab885bb6138969fc8f1 (127.25.130.3:46001): Not found: Tablet not found: f78b104c96db4a72839fc74e982a2ee7
I20260812 19:06:27.017812 12105 raft_consensus.cc:3037] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:27.020484 12105 raft_consensus.cc:2454] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7402118b06af4bb1bc693820faf92a6c in term 2.
I20260812 19:06:27.020862 11861 leader_election.cc:302] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 122a6c94061e4dc394286c9ca63162c4, 7402118b06af4bb1bc693820faf92a6c; no voters: 16ebfcb8b3dc4ab885bb6138969fc8f1
I20260812 19:06:27.021071 12326 raft_consensus.cc:2781] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:06:27.021278 12326 raft_consensus.cc:686] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [term 2 LEADER]: Becoming Leader. State: Replica: 7402118b06af4bb1bc693820faf92a6c, State: Running, Role: LEADER
I20260812 19:06:27.021700 12326 consensus_queue.cc:237] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } }
I20260812 19:06:27.022202 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:27.021815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.1:51795)
0812 19:06:27.022068 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:27.022121 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177}
I20260812 19:06:27.099865 12337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:27.099326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.3:43013)
0812 19:06:27.099655 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:27.099724 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":124,"threads_started":1}
I20260812 19:06:27.432811 12105 raft_consensus.cc:1261] T f78b104c96db4a72839fc74e982a2ee7 P 122a6c94061e4dc394286c9ca63162c4 [term 2 FOLLOWER]: Refusing update from remote peer 7402118b06af4bb1bc693820faf92a6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260812 19:06:27.433810 12326 consensus_queue.cc:1035] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:27.436568 12340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:27.436053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.2:34659)
0812 19:06:27.436321 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:27.436406 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":118,"threads_started":1}
I20260812 19:06:27.439416 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:27.439122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.1:37419)
0812 19:06:27.439248 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:27.439312 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 19:06:27.439893 11863 consensus_peers.cc:487] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c -> Peer 16ebfcb8b3dc4ab885bb6138969fc8f1 (127.25.130.3:46001): Couldn't send request to peer 16ebfcb8b3dc4ab885bb6138969fc8f1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f78b104c96db4a72839fc74e982a2ee7. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:27.440344 12340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:27.440103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.2:43457)
0812 19:06:27.440212 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:27.440272 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 19:06:28.019867 12346 ts_tablet_manager.cc:927] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1: Initiating tablet copy from peer 7402118b06af4bb1bc693820faf92a6c (127.25.130.1:32921)
I20260812 19:06:28.020753 12346 tablet_copy_client.cc:316] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1: tablet copy: Beginning tablet copy session from remote peer at address 127.25.130.1:32921
Aug 12 19:06:28 dist-test-slave-49h3 krb5kdc[10388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561586, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for kudu/127.25.130.1@KRBTEST.COM
I20260812 19:06:28.032411 12350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:28.021487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:32921 (local address 127.25.130.3:43305)
0812 19:06:28.021842 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:28.021874 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:28.022081 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:28.022659 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:28.022666 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:28.023156 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:06:28.023310 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:28.023318 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:28.024487 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:28.024490 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:28.025251 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:28.025264 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:28.026018 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:28.026022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:28.026144 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:28.026802 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:06:28.026816 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:06:28.028609 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:06:28.030564 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:28.030576 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:28.030583 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:28.030850 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:28.031117 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:28.031119 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:28.031120 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:28.031199 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:28.031677 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:06:28.031684 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:06:28.031850 (+ 166us) client_negotiation.cc:769] Sending connection context
0812 19:06:28.032022 (+ 172us) client_negotiation.cc:240] Negotiation successful
0812 19:06:28.032231 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":160,"threads_started":1}
I20260812 19:06:28.032495 12351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:28.021703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:43305 (local address 127.25.130.1:32921)
0812 19:06:28.022058 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:28.022066 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:28.022096 (+ 30us) server_negotiation.cc:407] Connection header received
0812 19:06:28.022198 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:28.022204 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:28.022448 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:28.022596 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:28.023472 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:28.024318 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:28.025390 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:28.025863 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:28.025953 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:28.028752 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:28.028760 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:28.028775 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:28.028806 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:28.030452 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:28.030950 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:28.030956 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:28.030957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:28.031053 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:28.031261 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:28.031263 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:28.031264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:28.031588 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:28.031723 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:28.032092 (+ 369us) server_negotiation.cc:299] Negotiation successful
0812 19:06:28.032280 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":93,"threads_started":1}
I20260812 19:06:28.034042 11969 tablet_copy_service.cc:140] P 7402118b06af4bb1bc693820faf92a6c: Received BeginTabletCopySession request for tablet f78b104c96db4a72839fc74e982a2ee7 from peer 16ebfcb8b3dc4ab885bb6138969fc8f1 ({username='kudu', principal='kudu/127.25.130.3@KRBTEST.COM'} at 127.25.130.3:43305)
I20260812 19:06:28.034240 11969 tablet_copy_service.cc:161] P 7402118b06af4bb1bc693820faf92a6c: Beginning new tablet copy session on tablet f78b104c96db4a72839fc74e982a2ee7 from peer 16ebfcb8b3dc4ab885bb6138969fc8f1 at {username='kudu', principal='kudu/127.25.130.3@KRBTEST.COM'} at 127.25.130.3:43305: session id = 16ebfcb8b3dc4ab885bb6138969fc8f1-f78b104c96db4a72839fc74e982a2ee7
I20260812 19:06:28.035709 11969 tablet_copy_source_session.cc:205] T f78b104c96db4a72839fc74e982a2ee7 P 7402118b06af4bb1bc693820faf92a6c: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:06:28.040992 12346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f78b104c96db4a72839fc74e982a2ee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:28.049719 12346 tablet_copy_client.cc:791] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1: tablet copy: Starting download of 0 data blocks...
I20260812 19:06:28.049970 12346 tablet_copy_client.cc:655] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1: tablet copy: Starting download of 1 WAL segments...
I20260812 19:06:28.052249 12346 tablet_copy_client.cc:531] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:06:28.054454 12346 tablet_bootstrap.cc:492] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1: Bootstrap starting.
I20260812 19:06:28.061475 12346 log.cc:824] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1: Log is configured to *not* fsync() on all Append() calls
I20260812 19:06:28.070029 12346 tablet_bootstrap.cc:492] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:06:28.070405 12346 tablet_bootstrap.cc:492] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1: Bootstrap complete.
I20260812 19:06:28.070802 12346 ts_tablet_manager.cc:1397] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1: Time spent bootstrapping tablet: real 0.017s user 0.016s sys 0.000s
I20260812 19:06:28.073678 12346 raft_consensus.cc:348] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } }
I20260812 19:06:28.073974 12346 raft_consensus.cc:729] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16ebfcb8b3dc4ab885bb6138969fc8f1, State: Initialized, Role: FOLLOWER
I20260812 19:06:28.074523 12346 consensus_queue.cc:260] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16ebfcb8b3dc4ab885bb6138969fc8f1" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46001 } } peers { permanent_uuid: "122a6c94061e4dc394286c9ca63162c4" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 41435 } } peers { permanent_uuid: "7402118b06af4bb1bc693820faf92a6c" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 32921 } }
I20260812 19:06:28.075639 12346 ts_tablet_manager.cc:1428] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 19:06:28.075999 12350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:28.075665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.3:53781)
0812 19:06:28.075821 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:28.075895 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 19:06:28.076584 11969 tablet_copy_service.cc:342] P 7402118b06af4bb1bc693820faf92a6c: Request end of tablet copy session 16ebfcb8b3dc4ab885bb6138969fc8f1-f78b104c96db4a72839fc74e982a2ee7 received from {username='kudu', principal='kudu/127.25.130.3@KRBTEST.COM'} at 127.25.130.3:43305
I20260812 19:06:28.076682 12350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:28.076457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.3:37931)
0812 19:06:28.076575 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:28.076623 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 19:06:28.076742 11969 tablet_copy_service.cc:433] P 7402118b06af4bb1bc693820faf92a6c: ending tablet copy session 16ebfcb8b3dc4ab885bb6138969fc8f1-f78b104c96db4a72839fc74e982a2ee7 on tablet f78b104c96db4a72839fc74e982a2ee7 with peer 16ebfcb8b3dc4ab885bb6138969fc8f1
I20260812 19:06:28.440873 12355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:28.440310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.1:41653)
0812 19:06:28.440620 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:28.440698 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":96,"threads_started":1}
I20260812 19:06:28.441591 12356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:28.441115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.2:44097)
0812 19:06:28.441398 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:28.441487 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":72,"threads_started":1}
I20260812 19:06:28.561502 12355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:28.561124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.1:47617)
0812 19:06:28.561270 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:28.561357 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 19:06:28.561821 12265 raft_consensus.cc:1203] T f78b104c96db4a72839fc74e982a2ee7 P 16ebfcb8b3dc4ab885bb6138969fc8f1 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260812 19:06:28.562883 12350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:28.562607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.3:48673)
0812 19:06:28.562724 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:28.562796 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 19:06:29.443053 12358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:29.442448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.2:42233)
0812 19:06:29.442834 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:29.442911 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":103,"threads_started":1}
I20260812 19:06:29.562974 12359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:29.562454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.1:48403)
0812 19:06:29.562762 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:29.562846 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":104,"threads_started":1}
I20260812 19:06:29.564076 12360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:29.563652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.3:34161)
0812 19:06:29.563899 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:29.563956 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":75,"threads_started":1}
I20260812 19:06:30.444634 12361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:30.444060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.2:47021)
0812 19:06:30.444422 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:30.444499 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":109,"threads_started":1}
I20260812 19:06:30.564615 12362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:30.563961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.1:36759)
0812 19:06:30.564394 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:30.564464 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":159,"threads_started":1}
I20260812 19:06:30.565502 12363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:30.564830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.3:48069)
0812 19:06:30.565251 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:30.565352 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":116,"threads_started":1}
I20260812 19:06:31.446611 12364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:31.446032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.2:41583)
0812 19:06:31.446395 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:31.446476 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":118,"threads_started":1}
I20260812 19:06:31.566540 12366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:31.565927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.1:57545)
0812 19:06:31.566292 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:31.566368 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":148,"threads_started":1}
I20260812 19:06:31.566975 12368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:31.566490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.3:45919)
0812 19:06:31.566809 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:31.566876 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":103,"threads_started":1}
I20260812 19:06:31.567371 12366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:31.567129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.1:56697)
0812 19:06:31.567250 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:31.567301 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 19:06:32.448122 12369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:32.447527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.2:56669)
0812 19:06:32.447867 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:32.447954 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":101,"threads_started":1}
I20260812 19:06:32.568334 12370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:32.567780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.3:40585)
0812 19:06:32.568101 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:32.568169 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":100,"threads_started":1}
I20260812 19:06:32.568638 12371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:32.568158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.1:59595)
0812 19:06:32.568430 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:32.568513 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":80,"threads_started":1}
I20260812 19:06:33.449718 12372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:33.449137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.2:39273)
0812 19:06:33.449514 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:33.449589 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":95,"threads_started":1}
I20260812 19:06:33.569746 12373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:33.569229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.3:55365)
0812 19:06:33.569577 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:33.569637 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":103,"threads_started":1}
I20260812 19:06:33.569897 12375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:33.569437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.1:40163)
0812 19:06:33.569716 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:33.569775 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":71,"threads_started":1}
I20260812 19:06:34.451128 12377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:34.450533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.2:38553)
0812 19:06:34.450914 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:34.450994 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":104,"threads_started":1}
I20260812 19:06:34.571098 12379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:34.570624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.1:51789)
0812 19:06:34.570926 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:34.571001 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":114,"threads_started":1}
I20260812 19:06:34.571099 12378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:34.570623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.3:59701)
0812 19:06:34.570910 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:34.570979 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":79,"threads_started":1}
I20260812 19:06:35.452440 12380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:35.451959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.2:34789)
0812 19:06:35.452270 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:35.452332 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":114,"threads_started":1}
I20260812 19:06:35.572451 12382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:35.571994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.3:48099)
0812 19:06:35.572286 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:35.572349 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":102,"threads_started":1}
I20260812 19:06:35.572451 12381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:35.571995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.1:54515)
0812 19:06:35.572285 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:35.572348 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":79,"threads_started":1}
I20260812 19:06:36.453752 12384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:36.453274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.2:46299)
0812 19:06:36.453576 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:36.453640 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":79,"threads_started":1}
I20260812 19:06:36.573807 12385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:36.573327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.3:60949)
0812 19:06:36.573618 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:36.573699 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":78,"threads_started":1}
I20260812 19:06:36.573840 12386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:36.573421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.1:50763)
0812 19:06:36.573698 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:36.573750 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":80,"threads_started":1}
I20260812 19:06:37.455137 12387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:37.454604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.2:33543)
0812 19:06:37.454931 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:37.454999 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":104,"threads_started":1}
I20260812 19:06:37.575384 12389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:37.574763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.1:52603)
0812 19:06:37.575210 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:37.575273 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":130,"threads_started":1}
I20260812 19:06:37.575364 12388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:37.574761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.3:60543)
0812 19:06:37.575119 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:37.575193 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":125,"threads_started":1}
I20260812 19:06:38.456877 12390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:38.456191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.2:43257)
0812 19:06:38.456679 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:38.456741 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":126,"threads_started":1}
I20260812 19:06:38.577124 12394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:38.576435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.1:34637)
0812 19:06:38.576882 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:38.576960 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":122,"threads_started":1}
I20260812 19:06:38.577126 12391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:38.576435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.3:45813)
0812 19:06:38.576864 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:38.576962 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":121,"threads_started":1}
I20260812 19:06:39.458534 12395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:39.457931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.2:49757)
0812 19:06:39.458295 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:39.458374 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":102,"threads_started":1}
I20260812 19:06:39.578819 12397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:39.578231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.1:45137)
0812 19:06:39.578652 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:39.578711 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":109,"threads_started":1}
I20260812 19:06:39.578821 12396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:39.578230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.3:46555)
0812 19:06:39.578580 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:39.578665 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":103,"threads_started":1}
I20260812 19:06:40.460052 12398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:40.459496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.2:49433)
0812 19:06:40.459842 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:40.459913 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":94,"threads_started":1}
I20260812 19:06:40.580220 12399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:40.579736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.3:47551)
0812 19:06:40.580029 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:40.580099 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":87,"threads_started":1}
I20260812 19:06:40.580273 12400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:40.579737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:43125 (local address 127.25.130.1:60211)
0812 19:06:40.580090 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:40.580149 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":91,"threads_started":1}
I20260812 19:06:41.108456 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 11801
I20260812 19:06:41.118737 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 12004
I20260812 19:06:41.129041 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 12164
2026-08-12T19:06:41Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19460 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:06:41 dist-test-slave-49h3 krb5kdc[12405](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:06:41 dist-test-slave-49h3 krb5kdc[12405](info): set up 2 sockets
Aug 12 19:06:41 dist-test-slave-49h3 krb5kdc[12405](info): commencing operation
krb5kdc: starting...
W20260812 19:06:42.200660 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:06:42 dist-test-slave-49h3 krb5kdc[12405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:06:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:06:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:06:42.340183 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:46215
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:36571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:46215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:06:42.472131 12421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:06:42.472462 12421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:06:42.472528 12421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:06:42.476624 12421 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 19:06:42.476672 12421 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 19:06:42.476693 12421 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 19:06:42.476711 12421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:06:42.476737 12421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:06:42.476768 12421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:06:42.476799 12421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:06:42.476827 12421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:06:42.482278 12421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:36571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:46215
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:46215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:06:42.483561 12421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:06:42.484795 12421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:42.494903 12430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:42.494951 12427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:42.494999 12421 server_base.cc:1034] running on GCE node
W20260812 19:06:42.494918 12428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:42.495808 12421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:06:42.497097 12421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:06:42.498263 12421 hybrid_clock.cc:648] HybridClock initialized: now 1786561602498244 us; error 32 us; skew 500 ppm
Aug 12 19:06:42 dist-test-slave-49h3 krb5kdc[12405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561602, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:42.506616 12421 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:06:42.507887 12421 webserver.cc:460] Webserver started at http://127.25.130.62:38947/ using document root <none> and password file <none>
I20260812 19:06:42.508579 12421 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:42.508646 12421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:42.508899 12421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:06:42.511492 12421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "3f33a968c92f4a55b07f34cfee721875"
format_stamp: "Formatted at 2026-08-12 19:06:42 on dist-test-slave-49h3"
server_key: "5a77b2e4618563069ebf0b91b01060b2"
server_key_iv: "cdcf42f59f6fe7fda2175a43131c3361"
server_key_version: "encryptionkey@0"
I20260812 19:06:42.512141 12421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "3f33a968c92f4a55b07f34cfee721875"
format_stamp: "Formatted at 2026-08-12 19:06:42 on dist-test-slave-49h3"
server_key: "5a77b2e4618563069ebf0b91b01060b2"
server_key_iv: "cdcf42f59f6fe7fda2175a43131c3361"
server_key_version: "encryptionkey@0"
I20260812 19:06:42.516543 12421 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 19:06:42.519522 12439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:42.521188 12421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 19:06:42.521384 12421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "3f33a968c92f4a55b07f34cfee721875"
format_stamp: "Formatted at 2026-08-12 19:06:42 on dist-test-slave-49h3"
server_key: "5a77b2e4618563069ebf0b91b01060b2"
server_key_iv: "cdcf42f59f6fe7fda2175a43131c3361"
server_key_version: "encryptionkey@0"
I20260812 19:06:42.521567 12421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:42.535750 12421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:42.539855 12421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:06:42.540083 12421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:42.548858 12421 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:46215
I20260812 19:06:42.549201 12504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:46215 every 8 connection(s)
I20260812 19:06:42.550107 12421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:06:42.553700 12505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:42.559056 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 12421
I20260812 19:06:42.559219 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:06:42.559563 26120 external_mini_cluster.cc:1428] Setting key 705d98ce4baf492cb49521bb9a3a4a98
I20260812 19:06:42.561633 12505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875: Bootstrap starting.
I20260812 19:06:42.565107 12505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:42.566229 12505 log.cc:824] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875: Log is configured to *not* fsync() on all Append() calls
I20260812 19:06:42.569113 12505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875: No bootstrap required, opened a new log
Aug 12 19:06:42 dist-test-slave-49h3 krb5kdc[12405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:06:42.574365 12505 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f33a968c92f4a55b07f34cfee721875" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46215 } }
I20260812 19:06:42.574688 12505 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:42.574759 12505 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f33a968c92f4a55b07f34cfee721875, State: Initialized, Role: FOLLOWER
I20260812 19:06:42.575393 12505 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f33a968c92f4a55b07f34cfee721875" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46215 } }
I20260812 19:06:42.575609 12505 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:06:42.575685 12505 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:06:42.575796 12505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:42.577309 12505 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f33a968c92f4a55b07f34cfee721875" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46215 } }
I20260812 19:06:42.577790 12505 leader_election.cc:302] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f33a968c92f4a55b07f34cfee721875; no voters:
I20260812 19:06:42.578200 12505 leader_election.cc:288] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:06:42.578243 12509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:42.561394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35676 (local address 127.25.130.62:46215)
0812 19:06:42.561972 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:42.562003 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:42.562056 (+ 53us) server_negotiation.cc:407] Connection header received
0812 19:06:42.563088 (+ 1032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:42.563123 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:42.563671 (+ 548us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:42.564105 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:42.564890 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:42.566514 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:42.567718 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:42.568455 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:42.568632 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:42.571734 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:42.571743 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:42.571767 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:42.571817 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:42.574161 (+ 2344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:42.574773 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:42.574791 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:42.574798 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:42.574972 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:42.575289 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:42.575292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:42.575294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:42.575980 (+ 686us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:42.576199 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:42.576522 (+ 323us) server_negotiation.cc:299] Negotiation successful
0812 19:06:42.576938 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":163,"threads_started":1}
I20260812 19:06:42.578341 12511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:42.578586 12511 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [term 1 LEADER]: Becoming Leader. State: Replica: 3f33a968c92f4a55b07f34cfee721875, State: Running, Role: LEADER
I20260812 19:06:42.578992 12511 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f33a968c92f4a55b07f34cfee721875" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46215 } }
I20260812 19:06:42.579777 12505 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:06:42.580999 12512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f33a968c92f4a55b07f34cfee721875" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f33a968c92f4a55b07f34cfee721875" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46215 } } }
I20260812 19:06:42.581138 12512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [sys.catalog]: This master's current role is: LEADER
I20260812 19:06:42.581601 12513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f33a968c92f4a55b07f34cfee721875. Latest consensus state: current_term: 1 leader_uuid: "3f33a968c92f4a55b07f34cfee721875" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f33a968c92f4a55b07f34cfee721875" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46215 } } }
I20260812 19:06:42.581779 12513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [sys.catalog]: This master's current role is: LEADER
I20260812 19:06:42.582109 12520 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:06:42.586040 12520 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:06:42.592972 12520 catalog_manager.cc:1347] Generated new cluster ID: 89d5521c97c94a16b0ca3583b2bccb36
I20260812 19:06:42.593056 12520 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:06:42.629910 12520 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:06:42.631093 12520 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:06:42.648744 12520 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875: Generated new TSK 0
I20260812 19:06:42.649744 12520 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
I20260812 19:06:42.698489 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:0
--local_ip_for_outbound_sockets=127.25.130.1
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:46215
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:36571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:06:42.846784 12534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:06:42.847106 12534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:06:42.847148 12534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:06:42.851115 12534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:06:42.851176 12534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:06:42.851233 12534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:06:42.856566 12534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:36571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:46215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:06:42.857789 12534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:06:42.859054 12534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:42.870970 12545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:42.870955 12543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:42.870952 12542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:42.871018 12534 server_base.cc:1034] running on GCE node
I20260812 19:06:42.871708 12534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:06:42.872471 12534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:06:42.873674 12534 hybrid_clock.cc:648] HybridClock initialized: now 1786561602873651 us; error 35 us; skew 500 ppm
Aug 12 19:06:42 dist-test-slave-49h3 krb5kdc[12405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561602, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:42.876842 12534 init.cc:377] Logged in from keytab as kudu/127.25.130.1@KRBTEST.COM (short username kudu)
I20260812 19:06:42.877918 12534 webserver.cc:460] Webserver started at http://127.25.130.1:40437/ using document root <none> and password file <none>
I20260812 19:06:42.878582 12534 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:42.878649 12534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:42.878862 12534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:06:42.881305 12534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/data/instance:
uuid: "eecd5f36e76c4532ab715479e18f7b76"
format_stamp: "Formatted at 2026-08-12 19:06:42 on dist-test-slave-49h3"
server_key: "c0d79c6e412dcbd20589eff096ab95df"
server_key_iv: "85e22406d604e11de37086e02b06af69"
server_key_version: "encryptionkey@0"
I20260812 19:06:42.882015 12534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance:
uuid: "eecd5f36e76c4532ab715479e18f7b76"
format_stamp: "Formatted at 2026-08-12 19:06:42 on dist-test-slave-49h3"
server_key: "c0d79c6e412dcbd20589eff096ab95df"
server_key_iv: "85e22406d604e11de37086e02b06af69"
server_key_version: "encryptionkey@0"
I20260812 19:06:42.894734 12534 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.010s sys 0.000s
I20260812 19:06:42.904995 12552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:42.911152 12534 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.008s sys 0.000s
I20260812 19:06:42.911587 12534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "eecd5f36e76c4532ab715479e18f7b76"
format_stamp: "Formatted at 2026-08-12 19:06:42 on dist-test-slave-49h3"
server_key: "c0d79c6e412dcbd20589eff096ab95df"
server_key_iv: "85e22406d604e11de37086e02b06af69"
server_key_version: "encryptionkey@0"
I20260812 19:06:42.911849 12534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:42.976194 12534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:42.987512 12534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:06:42.988029 12534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:42.990298 12534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:06:42.993505 12534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:06:42.993685 12534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:42.993790 12534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:06:42.993862 12534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:43.027659 12534 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:39005
I20260812 19:06:43.027997 12689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:39005 every 8 connection(s)
I20260812 19:06:43.031914 12534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:06:43.034444 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 12534
I20260812 19:06:43.034950 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance
I20260812 19:06:43.036046 26120 external_mini_cluster.cc:1428] Setting key eafdb6446b07e1f82fa3c5dabc81bff5
Aug 12 19:06:43 dist-test-slave-49h3 krb5kdc[12405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561602, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.130.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.2@KRBTEST.COM" created.
I20260812 19:06:43.085367 12509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:43.039164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:56119 (local address 127.25.130.62:46215)
0812 19:06:43.039664 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:43.039748 (+ 84us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:43.044528 (+ 4780us) server_negotiation.cc:407] Connection header received
0812 19:06:43.047449 (+ 2921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:43.047474 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:43.047675 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:43.047876 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:43.052422 (+ 4546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.054527 (+ 2105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.058835 (+ 4308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.059699 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.060143 (+ 444us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:43.070357 (+ 10214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:43.070376 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:43.070379 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:43.070510 (+ 131us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:43.076110 (+ 5600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:43.078353 (+ 2243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:43.078378 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:43.078386 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:43.078677 (+ 291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:43.079602 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:43.079618 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:43.079620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:43.080272 (+ 652us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:43.080614 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:43.084380 (+ 3766us) server_negotiation.cc:299] Negotiation successful
0812 19:06:43.085016 (+ 636us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260812 19:06:43.090626 12694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:43.041242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.1:56119)
0812 19:06:43.044113 (+ 2871us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:43.044227 (+ 114us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:43.046765 (+ 2538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:43.048571 (+ 1806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:43.048587 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:43.049905 (+ 1318us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:06:43.051999 (+ 2094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:43.052053 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.055020 (+ 2967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.055030 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:43.058385 (+ 3355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:43.058427 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.060356 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.060377 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:43.061122 (+ 745us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:43.063446 (+ 2324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:06:43.063521 (+ 75us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:06:43.069417 (+ 5896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:06:43.076634 (+ 7217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:43.076673 (+ 39us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:43.076763 (+ 90us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:43.077965 (+ 1202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:43.078961 (+ 996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:43.078964 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:43.078966 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:43.079393 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:43.080837 (+ 1444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:06:43.080849 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:06:43.082576 (+ 1727us) client_negotiation.cc:769] Sending connection context
0812 19:06:43.083780 (+ 1204us) client_negotiation.cc:240] Negotiation successful
0812 19:06:43.085914 (+ 2134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2090,"thread_start_us":540,"threads_started":1}
I20260812 19:06:43.096606 12690 heartbeater.cc:347] Connected to a master server at 127.25.130.62:46215
I20260812 19:06:43.097636 12690 heartbeater.cc:464] Registering TS with master...
I20260812 19:06:43.099593 12690 heartbeater.cc:511] Master 127.25.130.62:46215 requested a full tablet report, sending...
I20260812 19:06:43.104209 12461 ts_manager.cc:194] Registered new tserver with Master: eecd5f36e76c4532ab715479e18f7b76 (127.25.130.1:39005)
I20260812 19:06:43.108285 12461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.1@KRBTEST.COM'} at 127.25.130.1:56119
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
I20260812 19:06:43.120009 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:0
--local_ip_for_outbound_sockets=127.25.130.2
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:46215
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:36571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:06:43.437409 12697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:06:43.437727 12697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:06:43.437770 12697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:06:43.441789 12697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:06:43.441852 12697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:06:43.441911 12697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:06:43.447257 12697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:36571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:46215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:06:43.448352 12697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:06:43.449390 12697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:43.459493 12703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:43.459582 12702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:43.459502 12705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:43.459882 12697 server_base.cc:1034] running on GCE node
I20260812 19:06:43.460316 12697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:06:43.460994 12697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:06:43.462182 12697 hybrid_clock.cc:648] HybridClock initialized: now 1786561603462161 us; error 39 us; skew 500 ppm
Aug 12 19:06:43 dist-test-slave-49h3 krb5kdc[12405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561603, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:43.465086 12697 init.cc:377] Logged in from keytab as kudu/127.25.130.2@KRBTEST.COM (short username kudu)
I20260812 19:06:43.466141 12697 webserver.cc:460] Webserver started at http://127.25.130.2:42143/ using document root <none> and password file <none>
I20260812 19:06:43.466753 12697 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:43.466826 12697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:43.467060 12697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:06:43.469257 12697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/data/instance:
uuid: "152330aead374ba289c8b794390d7bbf"
format_stamp: "Formatted at 2026-08-12 19:06:43 on dist-test-slave-49h3"
server_key: "20e74f6a54c593565ef9b29ed53eae9c"
server_key_iv: "c828bcbf0a52ac50d62bdf0461090a67"
server_key_version: "encryptionkey@0"
I20260812 19:06:43.469909 12697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance:
uuid: "152330aead374ba289c8b794390d7bbf"
format_stamp: "Formatted at 2026-08-12 19:06:43 on dist-test-slave-49h3"
server_key: "20e74f6a54c593565ef9b29ed53eae9c"
server_key_iv: "c828bcbf0a52ac50d62bdf0461090a67"
server_key_version: "encryptionkey@0"
I20260812 19:06:43.473994 12697 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:06:43.476557 12714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:43.477903 12697 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:06:43.478037 12697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "152330aead374ba289c8b794390d7bbf"
format_stamp: "Formatted at 2026-08-12 19:06:43 on dist-test-slave-49h3"
server_key: "20e74f6a54c593565ef9b29ed53eae9c"
server_key_iv: "c828bcbf0a52ac50d62bdf0461090a67"
server_key_version: "encryptionkey@0"
I20260812 19:06:43.478155 12697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:43.504215 12697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:43.507617 12697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:06:43.507810 12697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:43.508460 12697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:06:43.509384 12697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:06:43.509435 12697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:43.509521 12697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:06:43.509546 12697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:43.518035 12697 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:38941
I20260812 19:06:43.518069 12848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:38941 every 8 connection(s)
I20260812 19:06:43.518913 12697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:06:43.526371 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 12697
I20260812 19:06:43.526507 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance
I20260812 19:06:43.526850 26120 external_mini_cluster.cc:1428] Setting key 0acd65407eefb97c74d398b4ff1484b6
Aug 12 19:06:43 dist-test-slave-49h3 krb5kdc[12405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561603, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:06:43.532550 12509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:43.520766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:56117 (local address 127.25.130.62:46215)
0812 19:06:43.520911 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:43.520915 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:43.522043 (+ 1128us) server_negotiation.cc:407] Connection header received
0812 19:06:43.522938 (+ 895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:43.522941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:43.523006 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:43.523088 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:43.524292 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.524868 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.526062 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.526328 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.526418 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:43.529127 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:43.529131 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:43.529133 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:43.529158 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:43.530530 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:43.531028 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:43.531031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:43.531033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:43.531110 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:43.531426 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:43.531429 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:43.531431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:43.531626 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:43.531716 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:43.532236 (+ 520us) server_negotiation.cc:299] Negotiation successful
0812 19:06:43.532371 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 19:06:43.533434 12851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:43.521127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.2:56117)
0812 19:06:43.521910 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:43.521953 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:43.522735 (+ 782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:43.523287 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:43.523294 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:43.523652 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:06:43.524147 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:43.524162 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.524990 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.524993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:43.525932 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:43.525943 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.526475 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.526480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:43.526715 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:43.527302 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:06:43.527319 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:06:43.528957 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:06:43.530652 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:43.530659 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:43.530674 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:43.530932 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:43.531193 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:43.531196 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:43.531198 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:43.531340 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:43.531727 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:06:43.531735 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:06:43.531989 (+ 254us) client_negotiation.cc:769] Sending connection context
0812 19:06:43.532175 (+ 186us) client_negotiation.cc:240] Negotiation successful
0812 19:06:43.532513 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":109,"threads_started":1}
I20260812 19:06:43.534616 12849 heartbeater.cc:347] Connected to a master server at 127.25.130.62:46215
I20260812 19:06:43.534893 12849 heartbeater.cc:464] Registering TS with master...
I20260812 19:06:43.535509 12849 heartbeater.cc:511] Master 127.25.130.62:46215 requested a full tablet report, sending...
I20260812 19:06:43.536674 12461 ts_manager.cc:194] Registered new tserver with Master: 152330aead374ba289c8b794390d7bbf (127.25.130.2:38941)
WARNING: no policy specified for kudu/127.25.130.3@KRBTEST.COM; defaulting to no policy
I20260812 19:06:43.537262 12461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.2@KRBTEST.COM'} at 127.25.130.2:56117
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
I20260812 19:06:43.554674 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:0
--local_ip_for_outbound_sockets=127.25.130.3
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:46215
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:36571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:06:43.666869 12854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:06:43.667167 12854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:06:43.667213 12854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:06:43.671348 12854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:06:43.671414 12854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:06:43.671475 12854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:06:43.676968 12854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:36571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:46215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:06:43.678150 12854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:06:43.679164 12854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:43.688868 12863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:43.688861 12860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:43.688907 12854 server_base.cc:1034] running on GCE node
W20260812 19:06:43.688855 12861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:43.689632 12854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:06:43.690284 12854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:06:43.691447 12854 hybrid_clock.cc:648] HybridClock initialized: now 1786561603691426 us; error 40 us; skew 500 ppm
Aug 12 19:06:43 dist-test-slave-49h3 krb5kdc[12405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561603, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:06:43.694149 12854 init.cc:377] Logged in from keytab as kudu/127.25.130.3@KRBTEST.COM (short username kudu)
I20260812 19:06:43.695125 12854 webserver.cc:460] Webserver started at http://127.25.130.3:46133/ using document root <none> and password file <none>
I20260812 19:06:43.695688 12854 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:43.695753 12854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:43.695960 12854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:06:43.698122 12854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/data/instance:
uuid: "470846748a814ae1bfaf876b0a52b9f0"
format_stamp: "Formatted at 2026-08-12 19:06:43 on dist-test-slave-49h3"
server_key: "90aa3c078c3a419be516eaabb56cb3fa"
server_key_iv: "e4a1395f8fa20c1cf4ab6e828fafe557"
server_key_version: "encryptionkey@0"
I20260812 19:06:43.698705 12854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance:
uuid: "470846748a814ae1bfaf876b0a52b9f0"
format_stamp: "Formatted at 2026-08-12 19:06:43 on dist-test-slave-49h3"
server_key: "90aa3c078c3a419be516eaabb56cb3fa"
server_key_iv: "e4a1395f8fa20c1cf4ab6e828fafe557"
server_key_version: "encryptionkey@0"
I20260812 19:06:43.702497 12854 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 19:06:43.704974 12871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:43.706372 12854 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 19:06:43.706486 12854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "470846748a814ae1bfaf876b0a52b9f0"
format_stamp: "Formatted at 2026-08-12 19:06:43 on dist-test-slave-49h3"
server_key: "90aa3c078c3a419be516eaabb56cb3fa"
server_key_iv: "e4a1395f8fa20c1cf4ab6e828fafe557"
server_key_version: "encryptionkey@0"
I20260812 19:06:43.706620 12854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:43.723064 12854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:43.725979 12854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:06:43.726164 12854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:43.726737 12854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:06:43.727612 12854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:06:43.727659 12854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:43.727698 12854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:06:43.727721 12854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:43.735838 12854 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:45501
I20260812 19:06:43.735891 12994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:45501 every 8 connection(s)
I20260812 19:06:43.736690 12854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:06:43.742388 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 12854
I20260812 19:06:43.742486 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance
I20260812 19:06:43.742753 26120 external_mini_cluster.cc:1428] Setting key ba80162da6106bb1cf3cc0819f4699d0
Aug 12 19:06:43 dist-test-slave-49h3 krb5kdc[12405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561603, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:06:43.749739 12509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:43.738452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:59933 (local address 127.25.130.62:46215)
0812 19:06:43.738593 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:43.738597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:43.739528 (+ 931us) server_negotiation.cc:407] Connection header received
0812 19:06:43.740348 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:43.740350 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:43.740397 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:43.740466 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:43.741636 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.742296 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.743652 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.743970 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.744086 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:43.746588 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:43.746591 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:43.746593 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:43.746619 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:43.747830 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:43.748372 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:43.748374 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:43.748376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:43.748451 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:43.748715 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:43.748717 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:43.748718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:43.748884 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:43.748955 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:43.749496 (+ 541us) server_negotiation.cc:299] Negotiation successful
0812 19:06:43.749617 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 19:06:43.750499 12998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:43.738752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.3:59933)
0812 19:06:43.739385 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:43.739427 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:43.740179 (+ 752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:43.740618 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:43.740624 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:43.740968 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:06:43.741477 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:43.741499 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.742428 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.742430 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:43.743522 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:43.743534 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.744139 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.744143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:43.744345 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:43.744976 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:06:43.744992 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:06:43.746436 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:06:43.747980 (+ 1544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:43.747986 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:43.747998 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:43.748284 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:43.748515 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:06:43.748518 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:06:43.748519 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:06:43.748649 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:06:43.748953 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:06:43.748958 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:06:43.749215 (+ 257us) client_negotiation.cc:769] Sending connection context
0812 19:06:43.749372 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 19:06:43.749685 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":106,"threads_started":1}
I20260812 19:06:43.751583 12996 heartbeater.cc:347] Connected to a master server at 127.25.130.62:46215
I20260812 19:06:43.751796 12996 heartbeater.cc:464] Registering TS with master...
I20260812 19:06:43.752265 12996 heartbeater.cc:511] Master 127.25.130.62:46215 requested a full tablet report, sending...
I20260812 19:06:43.753224 12461 ts_manager.cc:194] Registered new tserver with Master: 470846748a814ae1bfaf876b0a52b9f0 (127.25.130.3:45501)
I20260812 19:06:43.753795 12461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.3@KRBTEST.COM'} at 127.25.130.3:59933
I20260812 19:06:43.756407 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:06:43.765059 12509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:43.758312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35692 (local address 127.25.130.62:46215)
0812 19:06:43.758437 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:43.758441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:43.758613 (+ 172us) server_negotiation.cc:407] Connection header received
0812 19:06:43.758763 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:43.758767 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:43.758810 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:43.758869 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:06:43.759543 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.760042 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.760842 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.761064 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.761154 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:43.762417 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:06:43.762420 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:06:43.762423 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:06:43.762448 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:06:43.763516 (+ 1068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:43.763899 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:43.763903 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:43.763905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:43.763985 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:06:43.764217 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:06:43.764220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:06:43.764222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:06:43.764440 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:06:43.764556 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:43.764798 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 19:06:43.764908 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 19:06:43.767850 12461 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:06:43.771188 12461 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:06:43.785629 13011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:43.779789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:45501 (local address 127.0.0.1:47116)
0812 19:06:43.780354 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:43.780408 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:43.780534 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:43.781181 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:43.781187 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:43.781212 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:06:43.781379 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:43.781387 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.782977 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.782979 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:43.783927 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:43.783936 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.785029 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.785033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:43.785164 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:43.785211 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 19:06:43.785320 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 19:06:43.785466 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":19,"thread_start_us":129,"threads_started":1}
I20260812 19:06:43.785668 13013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:43.779948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47116 (local address 127.25.130.3:45501)
0812 19:06:43.780624 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:43.780637 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:43.780664 (+ 27us) server_negotiation.cc:407] Connection header received
0812 19:06:43.780742 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:43.780748 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:43.781003 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:43.781607 (+ 604us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:06:43.781761 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.782859 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.784053 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.784901 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.785001 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:43.785166 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:43.785364 (+ 198us) server_negotiation.cc:299] Negotiation successful
0812 19:06:43.785508 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":180,"threads_started":1}
I20260812 19:06:43.786739 13012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:43.780577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:38941 (local address 127.0.0.1:35828)
0812 19:06:43.781491 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:43.781503 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:43.781648 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:43.782393 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:43.782396 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:43.782414 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:06:43.782497 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:43.782503 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.783832 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.783835 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:43.784976 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:43.784987 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.786325 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.786329 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:43.786426 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:43.786449 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 19:06:43.786519 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 19:06:43.786594 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":834,"mutex_wait_us":80,"thread_start_us":91,"threads_started":1}
I20260812 19:06:43.786782 13015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:43.780758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35828 (local address 127.25.130.2:38941)
0812 19:06:43.781276 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:43.781288 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:43.781578 (+ 290us) server_negotiation.cc:407] Connection header received
0812 19:06:43.781789 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:43.781795 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:43.781967 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:43.782183 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:06:43.782598 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.783693 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.785107 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.786214 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.786312 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:43.786447 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:43.786555 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 19:06:43.786654 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":119,"threads_started":1}
I20260812 19:06:43.789624 13014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:43.779923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:39005 (local address 127.0.0.1:55776)
0812 19:06:43.781223 (+ 1300us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:43.781236 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:43.781308 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:43.782163 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:43.782167 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:43.782185 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:06:43.782268 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:43.782273 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.783812 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.783816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:43.785001 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:43.785012 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.786300 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.786303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:43.786401 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:43.786424 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 19:06:43.786489 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 19:06:43.786559 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1239,"mutex_wait_us":36,"thread_start_us":91,"threads_started":1}
I20260812 19:06:43.789263 12766 tablet_service.cc:1467] Processing CreateTablet for tablet b38c8d3e99094f3691bc449c0f4b86d4 (DEFAULT_TABLE table=test-table [id=c147a51ac93e41aa80a8bebf48f49df0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:06:43.789081 12923 tablet_service.cc:1467] Processing CreateTablet for tablet b38c8d3e99094f3691bc449c0f4b86d4 (DEFAULT_TABLE table=test-table [id=c147a51ac93e41aa80a8bebf48f49df0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:06:43.790056 13016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:43.781060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55776 (local address 127.25.130.1:39005)
0812 19:06:43.781516 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:43.781533 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:43.781559 (+ 26us) server_negotiation.cc:407] Connection header received
0812 19:06:43.781644 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:43.781649 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:43.781842 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:43.781980 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:06:43.782380 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.783671 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.785149 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.785963 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.786093 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:43.786236 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:43.789791 (+ 3555us) server_negotiation.cc:299] Negotiation successful
0812 19:06:43.789910 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":125,"threads_started":1}
I20260812 19:06:43.790351 12766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b38c8d3e99094f3691bc449c0f4b86d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:43.791035 12923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b38c8d3e99094f3691bc449c0f4b86d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:43.792668 12604 tablet_service.cc:1467] Processing CreateTablet for tablet b38c8d3e99094f3691bc449c0f4b86d4 (DEFAULT_TABLE table=test-table [id=c147a51ac93e41aa80a8bebf48f49df0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:06:43.793612 12604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b38c8d3e99094f3691bc449c0f4b86d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:43.798199 13019 tablet_bootstrap.cc:492] T b38c8d3e99094f3691bc449c0f4b86d4 P 470846748a814ae1bfaf876b0a52b9f0: Bootstrap starting.
I20260812 19:06:43.798631 13018 tablet_bootstrap.cc:492] T b38c8d3e99094f3691bc449c0f4b86d4 P 152330aead374ba289c8b794390d7bbf: Bootstrap starting.
I20260812 19:06:43.799476 13022 tablet_bootstrap.cc:492] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76: Bootstrap starting.
I20260812 19:06:43.801186 13018 tablet_bootstrap.cc:654] T b38c8d3e99094f3691bc449c0f4b86d4 P 152330aead374ba289c8b794390d7bbf: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:43.801369 13022 tablet_bootstrap.cc:654] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:43.802277 13022 log.cc:824] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76: Log is configured to *not* fsync() on all Append() calls
I20260812 19:06:43.803740 13019 tablet_bootstrap.cc:654] T b38c8d3e99094f3691bc449c0f4b86d4 P 470846748a814ae1bfaf876b0a52b9f0: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:43.804311 13022 tablet_bootstrap.cc:492] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76: No bootstrap required, opened a new log
I20260812 19:06:43.804450 13019 log.cc:824] T b38c8d3e99094f3691bc449c0f4b86d4 P 470846748a814ae1bfaf876b0a52b9f0: Log is configured to *not* fsync() on all Append() calls
I20260812 19:06:43.804504 13022 ts_tablet_manager.cc:1397] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 19:06:43.805300 13018 log.cc:824] T b38c8d3e99094f3691bc449c0f4b86d4 P 152330aead374ba289c8b794390d7bbf: Log is configured to *not* fsync() on all Append() calls
I20260812 19:06:43.806515 13019 tablet_bootstrap.cc:492] T b38c8d3e99094f3691bc449c0f4b86d4 P 470846748a814ae1bfaf876b0a52b9f0: No bootstrap required, opened a new log
I20260812 19:06:43.806725 13019 ts_tablet_manager.cc:1397] T b38c8d3e99094f3691bc449c0f4b86d4 P 470846748a814ae1bfaf876b0a52b9f0: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260812 19:06:43.807397 13018 tablet_bootstrap.cc:492] T b38c8d3e99094f3691bc449c0f4b86d4 P 152330aead374ba289c8b794390d7bbf: No bootstrap required, opened a new log
I20260812 19:06:43.807580 13018 ts_tablet_manager.cc:1397] T b38c8d3e99094f3691bc449c0f4b86d4 P 152330aead374ba289c8b794390d7bbf: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.002s
I20260812 19:06:43.808450 13022 raft_consensus.cc:348] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } } peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } }
I20260812 19:06:43.808817 13022 raft_consensus.cc:374] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:43.808924 13022 raft_consensus.cc:729] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecd5f36e76c4532ab715479e18f7b76, State: Initialized, Role: FOLLOWER
I20260812 19:06:43.809547 13022 consensus_queue.cc:260] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } } peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } }
I20260812 19:06:43.810272 12690 heartbeater.cc:503] Master 127.25.130.62:46215 was elected leader, sending a full tablet report...
I20260812 19:06:43.810726 13022 ts_tablet_manager.cc:1428] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 19:06:43.810544 13019 raft_consensus.cc:348] T b38c8d3e99094f3691bc449c0f4b86d4 P 470846748a814ae1bfaf876b0a52b9f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } } peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } }
I20260812 19:06:43.810818 13019 raft_consensus.cc:374] T b38c8d3e99094f3691bc449c0f4b86d4 P 470846748a814ae1bfaf876b0a52b9f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:43.810874 13019 raft_consensus.cc:729] T b38c8d3e99094f3691bc449c0f4b86d4 P 470846748a814ae1bfaf876b0a52b9f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 470846748a814ae1bfaf876b0a52b9f0, State: Initialized, Role: FOLLOWER
I20260812 19:06:43.811290 13019 consensus_queue.cc:260] T b38c8d3e99094f3691bc449c0f4b86d4 P 470846748a814ae1bfaf876b0a52b9f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } } peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } }
I20260812 19:06:43.811316 13018 raft_consensus.cc:348] T b38c8d3e99094f3691bc449c0f4b86d4 P 152330aead374ba289c8b794390d7bbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } } peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } }
I20260812 19:06:43.811569 13018 raft_consensus.cc:374] T b38c8d3e99094f3691bc449c0f4b86d4 P 152330aead374ba289c8b794390d7bbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:43.811636 13018 raft_consensus.cc:729] T b38c8d3e99094f3691bc449c0f4b86d4 P 152330aead374ba289c8b794390d7bbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 152330aead374ba289c8b794390d7bbf, State: Initialized, Role: FOLLOWER
I20260812 19:06:43.812047 13018 consensus_queue.cc:260] T b38c8d3e99094f3691bc449c0f4b86d4 P 152330aead374ba289c8b794390d7bbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } } peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } }
I20260812 19:06:43.812301 13019 ts_tablet_manager.cc:1428] T b38c8d3e99094f3691bc449c0f4b86d4 P 470846748a814ae1bfaf876b0a52b9f0: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 19:06:43.812691 12996 heartbeater.cc:503] Master 127.25.130.62:46215 was elected leader, sending a full tablet report...
I20260812 19:06:43.812702 12849 heartbeater.cc:503] Master 127.25.130.62:46215 was elected leader, sending a full tablet report...
I20260812 19:06:43.813874 13018 ts_tablet_manager.cc:1428] T b38c8d3e99094f3691bc449c0f4b86d4 P 152330aead374ba289c8b794390d7bbf: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260812 19:06:43.833830 13029 raft_consensus.cc:482] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:43.834098 13029 raft_consensus.cc:504] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } } peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } }
I20260812 19:06:43.835306 13029 leader_election.cc:288] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 470846748a814ae1bfaf876b0a52b9f0 (127.25.130.3:45501), 152330aead374ba289c8b794390d7bbf (127.25.130.2:38941)
I20260812 19:06:43.839367 13015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:43.835805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:40319 (local address 127.25.130.2:38941)
0812 19:06:43.835918 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:43.835922 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:43.836132 (+ 210us) server_negotiation.cc:407] Connection header received
0812 19:06:43.836302 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:43.836305 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:43.836351 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:43.836419 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:06:43.836730 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.837241 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.838417 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.838802 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.838908 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:43.838964 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:43.839162 (+ 198us) server_negotiation.cc:299] Negotiation successful
0812 19:06:43.839235 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 19:06:43.839401 13032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:43.835619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:45501 (local address 127.25.130.1:36363)
0812 19:06:43.836021 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:43.836033 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:43.836193 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:43.836440 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:43.836443 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:43.836466 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:06:43.836640 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:43.836645 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.837345 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.837349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:43.838291 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:43.838301 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.838931 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.838935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:43.839080 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:43.839098 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 19:06:43.839171 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 19:06:43.839235 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":105,"spinlock_wait_cycles":16384,"thread_start_us":145,"threads_started":1}
I20260812 19:06:43.839402 13013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:43.835696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:36363 (local address 127.25.130.3:45501)
0812 19:06:43.835829 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:43.835834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:43.836138 (+ 304us) server_negotiation.cc:407] Connection header received
0812 19:06:43.836309 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:43.836311 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:43.836353 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:43.836418 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:06:43.836753 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.837238 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.838420 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.838815 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.838921 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:43.838970 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:43.839204 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 19:06:43.839274 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 19:06:43.839411 13035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:43.835619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:38941 (local address 127.25.130.1:40319)
0812 19:06:43.836021 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:43.836034 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:06:43.836192 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:06:43.836438 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:06:43.836441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:06:43.836463 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:06:43.836623 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:43.836629 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.837346 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.837349 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:43.838295 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:06:43.838305 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.838919 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.838923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:06:43.839017 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:43.839069 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 19:06:43.839126 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 19:06:43.839193 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":56,"spinlock_wait_cycles":35328,"thread_start_us":103,"threads_started":1}
I20260812 19:06:43.840116 12947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b38c8d3e99094f3691bc449c0f4b86d4" candidate_uuid: "eecd5f36e76c4532ab715479e18f7b76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "470846748a814ae1bfaf876b0a52b9f0" is_pre_election: true
I20260812 19:06:43.840212 12793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b38c8d3e99094f3691bc449c0f4b86d4" candidate_uuid: "eecd5f36e76c4532ab715479e18f7b76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "152330aead374ba289c8b794390d7bbf" is_pre_election: true
I20260812 19:06:43.840433 12947 raft_consensus.cc:2454] T b38c8d3e99094f3691bc449c0f4b86d4 P 470846748a814ae1bfaf876b0a52b9f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eecd5f36e76c4532ab715479e18f7b76 in term 0.
I20260812 19:06:43.840504 12793 raft_consensus.cc:2454] T b38c8d3e99094f3691bc449c0f4b86d4 P 152330aead374ba289c8b794390d7bbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eecd5f36e76c4532ab715479e18f7b76 in term 0.
I20260812 19:06:43.840953 12555 leader_election.cc:302] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 470846748a814ae1bfaf876b0a52b9f0, eecd5f36e76c4532ab715479e18f7b76; no voters:
I20260812 19:06:43.841236 13029 raft_consensus.cc:2781] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:43.841331 13029 raft_consensus.cc:482] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:43.841377 13029 raft_consensus.cc:3037] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:43.842514 13029 raft_consensus.cc:504] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } } peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } }
I20260812 19:06:43.842847 13029 leader_election.cc:288] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76 [CANDIDATE]: Term 1 election: Requested vote from peers 470846748a814ae1bfaf876b0a52b9f0 (127.25.130.3:45501), 152330aead374ba289c8b794390d7bbf (127.25.130.2:38941)
I20260812 19:06:43.843246 12947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b38c8d3e99094f3691bc449c0f4b86d4" candidate_uuid: "eecd5f36e76c4532ab715479e18f7b76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "470846748a814ae1bfaf876b0a52b9f0"
I20260812 19:06:43.843245 12793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b38c8d3e99094f3691bc449c0f4b86d4" candidate_uuid: "eecd5f36e76c4532ab715479e18f7b76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "152330aead374ba289c8b794390d7bbf"
I20260812 19:06:43.843379 12793 raft_consensus.cc:3037] T b38c8d3e99094f3691bc449c0f4b86d4 P 152330aead374ba289c8b794390d7bbf [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:43.843379 12947 raft_consensus.cc:3037] T b38c8d3e99094f3691bc449c0f4b86d4 P 470846748a814ae1bfaf876b0a52b9f0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:43.844633 12947 raft_consensus.cc:2454] T b38c8d3e99094f3691bc449c0f4b86d4 P 470846748a814ae1bfaf876b0a52b9f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecd5f36e76c4532ab715479e18f7b76 in term 1.
I20260812 19:06:43.844633 12793 raft_consensus.cc:2454] T b38c8d3e99094f3691bc449c0f4b86d4 P 152330aead374ba289c8b794390d7bbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecd5f36e76c4532ab715479e18f7b76 in term 1.
I20260812 19:06:43.844957 12555 leader_election.cc:302] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 470846748a814ae1bfaf876b0a52b9f0, eecd5f36e76c4532ab715479e18f7b76; no voters:
I20260812 19:06:43.845157 13029 raft_consensus.cc:2781] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:43.845376 13029 raft_consensus.cc:686] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76 [term 1 LEADER]: Becoming Leader. State: Replica: eecd5f36e76c4532ab715479e18f7b76, State: Running, Role: LEADER
I20260812 19:06:43.845721 13029 consensus_queue.cc:237] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } } peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } }
I20260812 19:06:43.848912 12460 catalog_manager.cc:5495] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76 reported cstate change: term changed from 0 to 1, leader changed from <none> to eecd5f36e76c4532ab715479e18f7b76 (127.25.130.1). New cstate: current_term: 1 leader_uuid: "eecd5f36e76c4532ab715479e18f7b76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:43.864740 13016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:43.861353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55782 (local address 127.25.130.1:39005)
0812 19:06:43.861521 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 19:06:43.861524 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:06:43.861546 (+ 22us) server_negotiation.cc:407] Connection header received
0812 19:06:43.861644 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:06:43.861646 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:06:43.861701 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:06:43.861788 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:06:43.862058 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.862577 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.863387 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:06:43.863602 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:06:43.863673 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:06:43.863910 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:06:43.864350 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:06:43.864441 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 19:06:43.864535 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 19:06:43.864588 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 19:06:43.871073 12793 raft_consensus.cc:1261] T b38c8d3e99094f3691bc449c0f4b86d4 P 152330aead374ba289c8b794390d7bbf [term 1 FOLLOWER]: Refusing update from remote peer eecd5f36e76c4532ab715479e18f7b76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:43.871097 12947 raft_consensus.cc:1261] T b38c8d3e99094f3691bc449c0f4b86d4 P 470846748a814ae1bfaf876b0a52b9f0 [term 1 FOLLOWER]: Refusing update from remote peer eecd5f36e76c4532ab715479e18f7b76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:43.871820 13029 consensus_queue.cc:1035] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:43.872022 13036 consensus_queue.cc:1035] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:43.885608 13041 mvcc.cc:205] Tried to move back new op lower bound from 7317756329447862272 to 7317756329354727424. Current Snapshot: MvccSnapshot[applied={T|T < 7317756329447862272}]
I20260812 19:06:43.885874 13040 mvcc.cc:205] Tried to move back new op lower bound from 7317756329447862272 to 7317756329354727424. Current Snapshot: MvccSnapshot[applied={T|T < 7317756329447862272}]
I20260812 19:06:43.889283 12460 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:06:43.889528 12460 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:06:43.892068 12460 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875: Sending DeleteTablet for 3 replicas of tablet b38c8d3e99094f3691bc449c0f4b86d4
I20260812 19:06:43.892940 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 12534
I20260812 19:06:43.892886 12923 tablet_service.cc:1514] Processing DeleteTablet for tablet b38c8d3e99094f3691bc449c0f4b86d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:06:43 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:47116
I20260812 19:06:43.893018 12766 tablet_service.cc:1514] Processing DeleteTablet for tablet b38c8d3e99094f3691bc449c0f4b86d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:06:43 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:35828
I20260812 19:06:43.898969 13050 tablet_replica.cc:330] T b38c8d3e99094f3691bc449c0f4b86d4 P 152330aead374ba289c8b794390d7bbf: stopping tablet replica
I20260812 19:06:43.899494 13050 raft_consensus.cc:2229] T b38c8d3e99094f3691bc449c0f4b86d4 P 152330aead374ba289c8b794390d7bbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:43.899891 13050 raft_consensus.cc:2258] T b38c8d3e99094f3691bc449c0f4b86d4 P 152330aead374ba289c8b794390d7bbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:43.900383 13049 tablet_replica.cc:330] T b38c8d3e99094f3691bc449c0f4b86d4 P 470846748a814ae1bfaf876b0a52b9f0: stopping tablet replica
I20260812 19:06:43.900714 13049 raft_consensus.cc:2229] T b38c8d3e99094f3691bc449c0f4b86d4 P 470846748a814ae1bfaf876b0a52b9f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:43.901028 13049 raft_consensus.cc:2258] T b38c8d3e99094f3691bc449c0f4b86d4 P 470846748a814ae1bfaf876b0a52b9f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:43.902626 13050 ts_tablet_manager.cc:1905] T b38c8d3e99094f3691bc449c0f4b86d4 P 152330aead374ba289c8b794390d7bbf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:06:43.902626 13049 ts_tablet_manager.cc:1905] T b38c8d3e99094f3691bc449c0f4b86d4 P 470846748a814ae1bfaf876b0a52b9f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:06:43.905424 13049 ts_tablet_manager.cc:1918] T b38c8d3e99094f3691bc449c0f4b86d4 P 470846748a814ae1bfaf876b0a52b9f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:06:43.905424 13050 ts_tablet_manager.cc:1918] T b38c8d3e99094f3691bc449c0f4b86d4 P 152330aead374ba289c8b794390d7bbf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:06:43.905556 13050 log.cc:1196] T b38c8d3e99094f3691bc449c0f4b86d4 P 152330aead374ba289c8b794390d7bbf: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/wal/wals/b38c8d3e99094f3691bc449c0f4b86d4
I20260812 19:06:43.905597 13049 log.cc:1196] T b38c8d3e99094f3691bc449c0f4b86d4 P 470846748a814ae1bfaf876b0a52b9f0: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/wal/wals/b38c8d3e99094f3691bc449c0f4b86d4
I20260812 19:06:43.905937 13049 ts_tablet_manager.cc:1939] T b38c8d3e99094f3691bc449c0f4b86d4 P 470846748a814ae1bfaf876b0a52b9f0: Deleting consensus metadata
I20260812 19:06:43.905937 13050 ts_tablet_manager.cc:1939] T b38c8d3e99094f3691bc449c0f4b86d4 P 152330aead374ba289c8b794390d7bbf: Deleting consensus metadata
W20260812 19:06:43.905946 12441 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.130.1:39005 (error 108)
I20260812 19:06:43.906266 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 12697
I20260812 19:06:43.907407 13014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:43.907114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:39005 (local address 127.0.0.1:55796)
0812 19:06:43.907231 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:43.907311 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.1:39005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 19:06:43.907425 12442 catalog_manager.cc:4841] TS 470846748a814ae1bfaf876b0a52b9f0 (127.25.130.3:45501): tablet b38c8d3e99094f3691bc449c0f4b86d4 (table test-table [id=c147a51ac93e41aa80a8bebf48f49df0]) successfully deleted
W20260812 19:06:43.907608 12441 catalog_manager.cc:4584] TS eecd5f36e76c4532ab715479e18f7b76 (127.25.130.1:39005): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b38c8d3e99094f3691bc449c0f4b86d4: Network error: Client connection negotiation failed: client connection to 127.25.130.1:39005: connect: Connection refused (error 111)
I20260812 19:06:43.915570 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 12854
I20260812 19:06:43.915967 13014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:06:43.915668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:38941 (local address 127.0.0.1:35836)
0812 19:06:43.915836 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 19:06:43.915883 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.2:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260812 19:06:43.922178 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 12421
I20260812 19:07:03.931734 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:46215
--webserver_interface=127.25.130.62
--webserver_port=38947
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:36571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:46215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:07:04.070498 13052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:04.070870 13052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:04.070945 13052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:04.075156 13052 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 19:07:04.075227 13052 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 19:07:04.075255 13052 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 19:07:04.075284 13052 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:07:04.075321 13052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:04.075361 13052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:04.075397 13052 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:07:04.075431 13052 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:07:04.081159 13052 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:36571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:46215
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:46215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=38947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:04.082548 13052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:04.083861 13052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:04.100108 13059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:04.100108 13063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:04.100109 13060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:04.100169 13052 server_base.cc:1034] running on GCE node
I20260812 19:07:04.101161 13052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:04.102607 13052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:04.103768 13052 hybrid_clock.cc:648] HybridClock initialized: now 1786561624103740 us; error 41 us; skew 500 ppm
Aug 12 19:07:04 dist-test-slave-49h3 krb5kdc[12405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561624, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:07:04.111665 13052 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:07:04.112797 13052 webserver.cc:460] Webserver started at http://127.25.130.62:38947/ using document root <none> and password file <none>
I20260812 19:07:04.113560 13052 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:04.113662 13052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:04.118537 13052 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 19:07:04.121094 13070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:04.122557 13052 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 19:07:04.122692 13052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "3f33a968c92f4a55b07f34cfee721875"
format_stamp: "Formatted at 2026-08-12 19:06:42 on dist-test-slave-49h3"
server_key: "5a77b2e4618563069ebf0b91b01060b2"
server_key_iv: "cdcf42f59f6fe7fda2175a43131c3361"
server_key_version: "encryptionkey@0"
I20260812 19:07:04.123441 13052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:04.151376 13052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:04.155515 13052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:04.155756 13052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:04.164686 13052 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:46215
I20260812 19:07:04.164996 13129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:46215 every 8 connection(s)
I20260812 19:07:04.166088 13052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:07:04.171737 13130 sys_catalog.cc:263] Verifying existing consensus state
I20260812 19:07:04.174234 13130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875: Bootstrap starting.
I20260812 19:07:04.174336 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 13052
I20260812 19:07:04.174809 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:39005
--local_ip_for_outbound_sockets=127.25.130.1
--tserver_master_addrs=127.25.130.62:46215
--webserver_port=40437
--webserver_interface=127.25.130.1
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:36571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:07:04.196622 13130 log.cc:824] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:04.206774 13130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:07:04.207222 13130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875: Bootstrap complete.
I20260812 19:07:04.211896 13130 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f33a968c92f4a55b07f34cfee721875" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46215 } }
I20260812 19:07:04.212349 13130 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f33a968c92f4a55b07f34cfee721875, State: Initialized, Role: FOLLOWER
I20260812 19:07:04.212911 13130 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f33a968c92f4a55b07f34cfee721875" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46215 } }
I20260812 19:07:04.213034 13130 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:07:04.213109 13130 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:07:04.213207 13130 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:04.215390 13130 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f33a968c92f4a55b07f34cfee721875" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46215 } }
I20260812 19:07:04.215758 13130 leader_election.cc:302] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f33a968c92f4a55b07f34cfee721875; no voters:
I20260812 19:07:04.216107 13130 leader_election.cc:288] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 19:07:04.216257 13134 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:04.216506 13134 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [term 2 LEADER]: Becoming Leader. State: Replica: 3f33a968c92f4a55b07f34cfee721875, State: Running, Role: LEADER
I20260812 19:07:04.216882 13134 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f33a968c92f4a55b07f34cfee721875" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46215 } }
I20260812 19:07:04.217571 13130 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:07:04.218410 13135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3f33a968c92f4a55b07f34cfee721875" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f33a968c92f4a55b07f34cfee721875" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46215 } } }
I20260812 19:07:04.218545 13135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [sys.catalog]: This master's current role is: LEADER
I20260812 19:07:04.219295 13136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f33a968c92f4a55b07f34cfee721875. Latest consensus state: current_term: 2 leader_uuid: "3f33a968c92f4a55b07f34cfee721875" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f33a968c92f4a55b07f34cfee721875" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46215 } } }
I20260812 19:07:04.219403 13136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [sys.catalog]: This master's current role is: LEADER
I20260812 19:07:04.219928 13141 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:07:04.224412 13141 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:07:04.224951 13141 catalog_manager.cc:1259] Loaded cluster ID: 89d5521c97c94a16b0ca3583b2bccb36
I20260812 19:07:04.225003 13141 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:07:04.227077 13141 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:07:04.227864 13141 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875: Loaded TSK: 0
I20260812 19:07:04.238435 13141 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875: Generated new TSK 1
I20260812 19:07:04.238729 13141 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 19:07:04.322388 13132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:04.322736 13132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:04.322803 13132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:04.326992 13132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:04.327064 13132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:04.327162 13132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:07:04.332669 13132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:36571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:39005
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=40437
--tserver_master_addrs=127.25.130.62:46215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:04.333962 13132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:04.335206 13132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:04.346081 13160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:04.346065 13163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:04.346396 13132 server_base.cc:1034] running on GCE node
W20260812 19:07:04.346400 13165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:04.346942 13132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:04.347642 13132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:04.348804 13132 hybrid_clock.cc:648] HybridClock initialized: now 1786561624348785 us; error 40 us; skew 500 ppm
Aug 12 19:07:04 dist-test-slave-49h3 krb5kdc[12405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561624, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:07:04.351821 13132 init.cc:377] Logged in from keytab as kudu/127.25.130.1@KRBTEST.COM (short username kudu)
I20260812 19:07:04.352787 13132 webserver.cc:460] Webserver started at http://127.25.130.1:40437/ using document root <none> and password file <none>
I20260812 19:07:04.353434 13132 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:04.353554 13132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:04.358047 13132 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260812 19:07:04.360347 13173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:04.361768 13132 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:07:04.361876 13132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "eecd5f36e76c4532ab715479e18f7b76"
format_stamp: "Formatted at 2026-08-12 19:06:42 on dist-test-slave-49h3"
server_key: "c0d79c6e412dcbd20589eff096ab95df"
server_key_iv: "85e22406d604e11de37086e02b06af69"
server_key_version: "encryptionkey@0"
I20260812 19:07:04.362504 13132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:04.374464 13132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:04.377671 13132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:04.377847 13132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:04.378518 13132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:07:04.379724 13181 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 19:07:04.384501 13132 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 19:07:04.384560 13132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260812 19:07:04.384617 13132 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 19:07:04.386641 13132 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 19:07:04.386686 13132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.001s
I20260812 19:07:04.386998 13181 tablet_bootstrap.cc:492] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76: Bootstrap starting.
I20260812 19:07:04.398238 13132 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:39005
I20260812 19:07:04.398332 13311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:39005 every 8 connection(s)
I20260812 19:07:04.399423 13132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:07:04.404021 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 13132
I20260812 19:07:04.404528 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:38941
--local_ip_for_outbound_sockets=127.25.130.2
--tserver_master_addrs=127.25.130.62:46215
--webserver_port=42143
--webserver_interface=127.25.130.2
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:36571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:07:04.411339 13181 log.cc:824] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76: Log is configured to *not* fsync() on all Append() calls
Aug 12 19:07:04 dist-test-slave-49h3 krb5kdc[12405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561624, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:07:04.420884 13181 tablet_bootstrap.cc:492] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:07:04.420995 13317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:04.402196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:50481 (local address 127.25.130.62:46215)
0812 19:07:04.402771 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:04.402793 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:04.402925 (+ 132us) server_negotiation.cc:407] Connection header received
0812 19:07:04.404684 (+ 1759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:04.404704 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:04.405138 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:04.405576 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:04.407431 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.408756 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.410201 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.410974 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.411151 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:04.414704 (+ 3553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:04.414711 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:04.414729 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:07:04.414767 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:04.416932 (+ 2165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:04.417637 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:04.417644 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:04.417652 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:04.417819 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:04.418230 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:04.418233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:04.418235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:04.418779 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:04.418920 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:04.419701 (+ 781us) server_negotiation.cc:299] Negotiation successful
0812 19:07:04.420018 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":153,"threads_started":1}
I20260812 19:07:04.421010 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:04.402029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.1:50481)
0812 19:07:04.402726 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:04.402791 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:04.404119 (+ 1328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:04.405668 (+ 1549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:04.405678 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:04.406254 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:07:04.407189 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:04.407216 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.408925 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.408928 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:04.410061 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:04.410074 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.411140 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.411145 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:04.411367 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:04.412275 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:07:04.412301 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:07:04.414472 (+ 2171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:07:04.417068 (+ 2596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:04.417075 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:04.417090 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:04.417518 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:04.417922 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:04.417925 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:04.417927 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:04.418111 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:04.418905 (+ 794us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:07:04.418912 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:07:04.419331 (+ 419us) client_negotiation.cc:769] Sending connection context
0812 19:07:04.419548 (+ 217us) client_negotiation.cc:240] Negotiation successful
0812 19:07:04.419919 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":181,"threads_started":1}
I20260812 19:07:04.421437 13181 tablet_bootstrap.cc:492] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76: Bootstrap complete.
I20260812 19:07:04.422981 13181 ts_tablet_manager.cc:1397] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76: Time spent bootstrapping tablet: real 0.036s user 0.026s sys 0.008s
I20260812 19:07:04.424062 13312 heartbeater.cc:347] Connected to a master server at 127.25.130.62:46215
I20260812 19:07:04.424321 13312 heartbeater.cc:464] Registering TS with master...
I20260812 19:07:04.424932 13312 heartbeater.cc:511] Master 127.25.130.62:46215 requested a full tablet report, sending...
I20260812 19:07:04.425923 13181 raft_consensus.cc:348] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } } peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } }
I20260812 19:07:04.426501 13181 raft_consensus.cc:729] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecd5f36e76c4532ab715479e18f7b76, State: Initialized, Role: FOLLOWER
I20260812 19:07:04.427057 13181 consensus_queue.cc:260] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } } peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } }
I20260812 19:07:04.428167 13090 ts_manager.cc:194] Registered new tserver with Master: eecd5f36e76c4532ab715479e18f7b76 (127.25.130.1:39005)
I20260812 19:07:04.428239 13181 ts_tablet_manager.cc:1428] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 19:07:04.432662 13090 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.1@KRBTEST.COM'} at 127.25.130.1:50481
I20260812 19:07:04.435647 13312 heartbeater.cc:503] Master 127.25.130.62:46215 was elected leader, sending a full tablet report...
Aug 12 19:07:04 dist-test-slave-49h3 krb5kdc[12405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561624, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for kudu/127.25.130.1@KRBTEST.COM
I20260812 19:07:04.442492 13323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:04.431819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:39005 (local address 127.0.0.1:55962)
0812 19:07:04.432433 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:04.432482 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:04.432632 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:04.433148 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:04.433155 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:04.433589 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:07:04.433715 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:04.433722 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.435035 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.435039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:04.435986 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:04.435998 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.436652 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.436656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:04.436778 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:04.437535 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:07:04.437547 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:07:04.438946 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:07:04.440715 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:04.440725 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:04.440730 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:04.440959 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:04.441243 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:04.441246 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:04.441248 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:04.441330 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:04.441810 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:07:04.441817 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:07:04.441941 (+ 124us) client_negotiation.cc:769] Sending connection context
0812 19:07:04.442066 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 19:07:04.442280 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":130,"threads_started":1}
I20260812 19:07:04.442492 13324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:04.431914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55962 (local address 127.25.130.1:39005)
0812 19:07:04.432673 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:04.432686 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:04.432710 (+ 24us) server_negotiation.cc:407] Connection header received
0812 19:07:04.432777 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:04.432782 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:04.432990 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:04.433123 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:04.433857 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.434905 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.436126 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.436540 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.436640 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:04.439080 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:04.439092 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:04.439102 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:07:04.439124 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:04.440609 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:04.441067 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:04.441072 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:04.441073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:04.441158 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:04.441409 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:04.441413 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:04.441415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:04.441697 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:04.441822 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:04.442163 (+ 341us) server_negotiation.cc:299] Negotiation successful
0812 19:07:04.442333 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":117,"threads_started":1}
I20260812 19:07:04.443576 13226 tablet_service.cc:1514] Processing DeleteTablet for tablet b38c8d3e99094f3691bc449c0f4b86d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:06:43 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:55962
I20260812 19:07:04.444725 13326 tablet_replica.cc:330] stopping tablet replica
I20260812 19:07:04.445042 13326 raft_consensus.cc:2229] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:04.445302 13326 raft_consensus.cc:2258] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:04.446456 13326 ts_tablet_manager.cc:1905] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:07:04.450547 13326 ts_tablet_manager.cc:1918] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:07:04.450646 13326 log.cc:1196] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/wal/wals/b38c8d3e99094f3691bc449c0f4b86d4
I20260812 19:07:04.450896 13326 ts_tablet_manager.cc:1939] T b38c8d3e99094f3691bc449c0f4b86d4 P eecd5f36e76c4532ab715479e18f7b76: Deleting consensus metadata
I20260812 19:07:04.453258 13072 catalog_manager.cc:4841] TS eecd5f36e76c4532ab715479e18f7b76 (127.25.130.1:39005): tablet b38c8d3e99094f3691bc449c0f4b86d4 (table test-table [id=c147a51ac93e41aa80a8bebf48f49df0]) successfully deleted
W20260812 19:07:04.538803 13318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:04.539083 13318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:04.539129 13318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:04.543159 13318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:04.543210 13318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:04.543274 13318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:07:04.548496 13318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:36571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:38941
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=42143
--tserver_master_addrs=127.25.130.62:46215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:04.549685 13318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:04.550882 13318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:04.561967 13332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:04.561985 13335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:04.562031 13331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:04.562029 13318 server_base.cc:1034] running on GCE node
I20260812 19:07:04.562825 13318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:04.563509 13318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:04.564668 13318 hybrid_clock.cc:648] HybridClock initialized: now 1786561624564651 us; error 27 us; skew 500 ppm
Aug 12 19:07:04 dist-test-slave-49h3 krb5kdc[12405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561624, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:07:04.567619 13318 init.cc:377] Logged in from keytab as kudu/127.25.130.2@KRBTEST.COM (short username kudu)
I20260812 19:07:04.568588 13318 webserver.cc:460] Webserver started at http://127.25.130.2:42143/ using document root <none> and password file <none>
I20260812 19:07:04.569239 13318 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:04.569334 13318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:04.573767 13318 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 19:07:04.576018 13342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:04.577339 13318 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:07:04.577479 13318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "152330aead374ba289c8b794390d7bbf"
format_stamp: "Formatted at 2026-08-12 19:06:43 on dist-test-slave-49h3"
server_key: "20e74f6a54c593565ef9b29ed53eae9c"
server_key_iv: "c828bcbf0a52ac50d62bdf0461090a67"
server_key_version: "encryptionkey@0"
I20260812 19:07:04.578102 13318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:04.590929 13318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:04.594503 13318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:04.594719 13318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:04.595484 13318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:07:04.596515 13318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:07:04.596570 13318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:04.596628 13318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:07:04.596657 13318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:04.606729 13318 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:38941
I20260812 19:07:04.606750 13474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:38941 every 8 connection(s)
I20260812 19:07:04.607911 13318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:07:04.614864 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 13318
I20260812 19:07:04.615320 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:45501
--local_ip_for_outbound_sockets=127.25.130.3
--tserver_master_addrs=127.25.130.62:46215
--webserver_port=46133
--webserver_interface=127.25.130.3
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:36571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 19:07:04 dist-test-slave-49h3 krb5kdc[12405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561624, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:07:04.623032 13317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:04.610071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:34105 (local address 127.25.130.62:46215)
0812 19:07:04.610230 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:04.610234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:04.611443 (+ 1209us) server_negotiation.cc:407] Connection header received
0812 19:07:04.612488 (+ 1045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:04.612491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:04.612566 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:04.612649 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:04.614074 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.614679 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.615978 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.616249 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.616348 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:04.619235 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:04.619239 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:04.619242 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:07:04.619276 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:04.620773 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:04.621264 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:04.621268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:04.621270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:04.621374 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:04.621753 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:04.621757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:04.621759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:04.622036 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:04.622140 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:04.622648 (+ 508us) server_negotiation.cc:299] Negotiation successful
0812 19:07:04.622814 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 19:07:04.623842 13477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:04.610530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.2:34105)
0812 19:07:04.611281 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:04.611326 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:04.612251 (+ 925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:04.612864 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:04.612871 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:04.613286 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:07:04.613911 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:04.613926 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.614806 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.614809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:04.615856 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:04.615866 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.616405 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.616410 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:04.616592 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:04.617204 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:07:04.617222 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:07:04.619038 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:07:04.620883 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:04.620888 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:04.620900 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:04.621158 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:04.621480 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:04.621483 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:04.621485 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:04.621662 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:04.622132 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:07:04.622137 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:07:04.622400 (+ 263us) client_negotiation.cc:769] Sending connection context
0812 19:07:04.622557 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 19:07:04.622834 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":135,"threads_started":1}
I20260812 19:07:04.625084 13475 heartbeater.cc:347] Connected to a master server at 127.25.130.62:46215
I20260812 19:07:04.625397 13475 heartbeater.cc:464] Registering TS with master...
I20260812 19:07:04.626173 13475 heartbeater.cc:511] Master 127.25.130.62:46215 requested a full tablet report, sending...
I20260812 19:07:04.627483 13090 ts_manager.cc:194] Registered new tserver with Master: 152330aead374ba289c8b794390d7bbf (127.25.130.2:38941)
I20260812 19:07:04.628129 13090 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.2@KRBTEST.COM'} at 127.25.130.2:34105
W20260812 19:07:04.745653 13478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:04.745944 13478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:04.745986 13478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:04.750046 13478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:04.750113 13478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:04.750174 13478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:07:04.755659 13478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:36571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:45501
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=46133
--tserver_master_addrs=127.25.130.62:46215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:04.756907 13478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:04.758172 13478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:04.769044 13485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:04.769047 13487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:04.769078 13484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:04.769116 13478 server_base.cc:1034] running on GCE node
I20260812 19:07:04.769765 13478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:04.770490 13478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:04.771698 13478 hybrid_clock.cc:648] HybridClock initialized: now 1786561624771680 us; error 35 us; skew 500 ppm
Aug 12 19:07:04 dist-test-slave-49h3 krb5kdc[12405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561624, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:07:04.774703 13478 init.cc:377] Logged in from keytab as kudu/127.25.130.3@KRBTEST.COM (short username kudu)
I20260812 19:07:04.775722 13478 webserver.cc:460] Webserver started at http://127.25.130.3:46133/ using document root <none> and password file <none>
I20260812 19:07:04.776346 13478 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:04.776440 13478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:04.780766 13478 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 19:07:04.783197 13495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:04.784577 13478 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 19:07:04.784701 13478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "470846748a814ae1bfaf876b0a52b9f0"
format_stamp: "Formatted at 2026-08-12 19:06:43 on dist-test-slave-49h3"
server_key: "90aa3c078c3a419be516eaabb56cb3fa"
server_key_iv: "e4a1395f8fa20c1cf4ab6e828fafe557"
server_key_version: "encryptionkey@0"
I20260812 19:07:04.785286 13478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:04.806972 13478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:04.810226 13478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:04.810428 13478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:04.811100 13478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:07:04.812028 13478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:07:04.812086 13478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:04.812141 13478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:07:04.812170 13478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:04.821555 13478 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:45501
I20260812 19:07:04.821590 13634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:45501 every 8 connection(s)
I20260812 19:07:04.822742 13478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:07:04.825078 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 13478
Aug 12 19:07:04 dist-test-slave-49h3 krb5kdc[12405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561624, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:07:04.834090 13639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:04.825813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40716 (local address 127.25.130.62:46215)
0812 19:07:04.826093 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:04.826098 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:04.826306 (+ 208us) server_negotiation.cc:407] Connection header received
0812 19:07:04.826481 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:04.826484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:04.826545 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:04.826653 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:04.827505 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.828192 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.829180 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.829416 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.829543 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:04.830937 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:04.830941 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:04.830943 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:07:04.830969 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:04.832350 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:04.832769 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:04.832772 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:04.832774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:04.832862 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:04.833155 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:04.833158 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:04.833159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:04.833346 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:04.833609 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:04.833746 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 19:07:04.833898 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":218,"thread_start_us":103,"threads_started":1}
I20260812 19:07:04.837234 13317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:04.824807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:58651 (local address 127.25.130.62:46215)
0812 19:07:04.825015 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:04.825021 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:04.826256 (+ 1235us) server_negotiation.cc:407] Connection header received
0812 19:07:04.827354 (+ 1098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:04.827357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:04.827406 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:04.827476 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:04.828840 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.829353 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.830709 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.830926 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.831013 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:04.833706 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:04.833709 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:04.833712 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:07:04.833735 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:04.835188 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:04.835712 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:04.835715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:04.835716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:04.835793 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:04.836140 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:04.836143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:04.836145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:04.836324 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:04.836408 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:04.836950 (+ 542us) server_negotiation.cc:299] Negotiation successful
0812 19:07:04.837071 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":81}
I20260812 19:07:04.838188 13637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:04.825254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.3:58651)
0812 19:07:04.826092 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:04.826156 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:04.827144 (+ 988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:04.827686 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:04.827694 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:04.828111 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:07:04.828698 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:04.828715 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.829507 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.829510 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:04.830584 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:04.830595 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.831082 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.831087 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:04.831285 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:04.831821 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:07:04.831843 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:07:04.833542 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:07:04.835301 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:04.835308 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:04.835327 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:04.835602 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:04.835874 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:04.835878 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:04.835880 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:04.836039 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:04.836431 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:07:04.836439 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:07:04.836702 (+ 263us) client_negotiation.cc:769] Sending connection context
0812 19:07:04.836872 (+ 170us) client_negotiation.cc:240] Negotiation successful
0812 19:07:04.837236 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":163,"threads_started":1}
I20260812 19:07:04.839282 13635 heartbeater.cc:347] Connected to a master server at 127.25.130.62:46215
I20260812 19:07:04.839509 13635 heartbeater.cc:464] Registering TS with master...
I20260812 19:07:04.839996 13635 heartbeater.cc:511] Master 127.25.130.62:46215 requested a full tablet report, sending...
I20260812 19:07:04.841032 13090 ts_manager.cc:194] Registered new tserver with Master: 470846748a814ae1bfaf876b0a52b9f0 (127.25.130.3:45501)
I20260812 19:07:04.841670 13090 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.3@KRBTEST.COM'} at 127.25.130.3:58651
I20260812 19:07:04.847844 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:07:04.851604 13317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:04.848458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40728 (local address 127.25.130.62:46215)
0812 19:07:04.848581 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:04.848585 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:04.848769 (+ 184us) server_negotiation.cc:407] Connection header received
0812 19:07:04.848911 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:04.848913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:04.849015 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:04.849101 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:07:04.849294 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.849843 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.850658 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.850859 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.850925 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:04.851153 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:07:04.851296 (+ 143us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 19:07:04.851491 (+ 195us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40728: token expired
Metrics: {"server-negotiator.queue_time_us":41}
W20260812 19:07:04.851673 13317 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40728: token expired
W20260812 19:07:04.852100 13002 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.25.130.62:46215)
I20260812 19:07:04.859279 13317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:04.852451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40742 (local address 127.25.130.62:46215)
0812 19:07:04.852570 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:04.852575 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:04.852587 (+ 12us) server_negotiation.cc:407] Connection header received
0812 19:07:04.852682 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:04.852684 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:04.852724 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:04.852787 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:04.853372 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.853920 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.854702 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.854916 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.854997 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:04.856244 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:04.856248 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:04.856250 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:07:04.856277 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:04.857722 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:04.858137 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:04.858141 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:04.858143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:04.858225 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:04.858470 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:04.858473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:04.858475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:04.858658 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:04.858778 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:04.859004 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 19:07:04.859125 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 19:07:04.861865 13090 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40742:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:07:04.865433 13090 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:07:04.873941 13226 tablet_service.cc:1467] Processing CreateTablet for tablet 9e30ca75b9e147e3ad8ac1c45b7f01f0 (DEFAULT_TABLE table=test-table [id=8a92b630debf445cb6c9c3d283e5b4fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:07:04.874739 13226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e30ca75b9e147e3ad8ac1c45b7f01f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:04.877985 13181 tablet_bootstrap.cc:492] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76: Bootstrap starting.
I20260812 19:07:04.878185 13647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:04.872970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:45501 (local address 127.0.0.1:50438)
0812 19:07:04.873287 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:04.873300 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:04.873389 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:04.874102 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:04.874105 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:04.874119 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:04.874192 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:04.874198 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.875729 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.875732 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:04.876625 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:04.876636 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.877714 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.877718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:04.877816 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:04.877879 (+ 63us) client_negotiation.cc:769] Sending connection context
0812 19:07:04.877929 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 19:07:04.878024 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":26,"thread_start_us":108,"threads_started":1}
I20260812 19:07:04.878324 13323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:04.872629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:38941 (local address 127.0.0.1:48524)
0812 19:07:04.872815 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:04.872831 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:04.872910 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:04.873979 (+ 1069us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:04.873982 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:04.874001 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:04.874126 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:04.874133 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.875665 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.875669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:04.876868 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:04.876881 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.877983 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.877987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:04.878081 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:04.878104 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 19:07:04.878151 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 19:07:04.878227 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":3}
I20260812 19:07:04.878931 13646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:04.872851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48524 (local address 127.25.130.2:38941)
0812 19:07:04.873372 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:04.873386 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:04.873415 (+ 29us) server_negotiation.cc:407] Connection header received
0812 19:07:04.873543 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:04.873549 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:04.873807 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:04.873983 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:04.874265 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.875535 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.877006 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.877853 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.878475 (+ 622us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:04.878632 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:04.878689 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 19:07:04.878784 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":126,"threads_started":1}
I20260812 19:07:04.878931 13648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:04.873299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50438 (local address 127.25.130.3:45501)
0812 19:07:04.873681 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:04.873691 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:04.873710 (+ 19us) server_negotiation.cc:407] Connection header received
0812 19:07:04.873771 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:04.873775 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:04.873921 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:04.874332 (+ 411us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:04.874529 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.875613 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.876756 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:04.877565 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:04.878461 (+ 896us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:04.878631 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:04.878689 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 19:07:04.878784 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":203,"threads_started":1}
I20260812 19:07:04.878997 13181 tablet_bootstrap.cc:654] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:04.880816 13181 tablet_bootstrap.cc:492] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76: No bootstrap required, opened a new log
I20260812 19:07:04.880921 13181 ts_tablet_manager.cc:1397] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 19:07:04.881587 13181 raft_consensus.cc:348] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } } peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } }
I20260812 19:07:04.881738 13181 raft_consensus.cc:374] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:04.881796 13181 raft_consensus.cc:729] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecd5f36e76c4532ab715479e18f7b76, State: Initialized, Role: FOLLOWER
I20260812 19:07:04.881963 13181 consensus_queue.cc:260] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } } peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } }
I20260812 19:07:04.882323 13181 ts_tablet_manager.cc:1428] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 19:07:04.882309 13555 tablet_service.cc:1467] Processing CreateTablet for tablet 9e30ca75b9e147e3ad8ac1c45b7f01f0 (DEFAULT_TABLE table=test-table [id=8a92b630debf445cb6c9c3d283e5b4fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:07:04.882309 13401 tablet_service.cc:1467] Processing CreateTablet for tablet 9e30ca75b9e147e3ad8ac1c45b7f01f0 (DEFAULT_TABLE table=test-table [id=8a92b630debf445cb6c9c3d283e5b4fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:07:04.883476 13555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e30ca75b9e147e3ad8ac1c45b7f01f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:04.883505 13401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e30ca75b9e147e3ad8ac1c45b7f01f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:04.891196 13651 tablet_bootstrap.cc:492] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 152330aead374ba289c8b794390d7bbf: Bootstrap starting.
I20260812 19:07:04.893893 13651 tablet_bootstrap.cc:654] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 152330aead374ba289c8b794390d7bbf: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:04.893893 13653 tablet_bootstrap.cc:492] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 470846748a814ae1bfaf876b0a52b9f0: Bootstrap starting.
I20260812 19:07:04.894923 13651 log.cc:824] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 152330aead374ba289c8b794390d7bbf: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:04.896523 13653 tablet_bootstrap.cc:654] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 470846748a814ae1bfaf876b0a52b9f0: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:04.897182 13651 tablet_bootstrap.cc:492] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 152330aead374ba289c8b794390d7bbf: No bootstrap required, opened a new log
I20260812 19:07:04.897339 13653 log.cc:824] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 470846748a814ae1bfaf876b0a52b9f0: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:04.897398 13651 ts_tablet_manager.cc:1397] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 152330aead374ba289c8b794390d7bbf: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 19:07:04.899115 13653 tablet_bootstrap.cc:492] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 470846748a814ae1bfaf876b0a52b9f0: No bootstrap required, opened a new log
I20260812 19:07:04.899293 13653 ts_tablet_manager.cc:1397] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 470846748a814ae1bfaf876b0a52b9f0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 19:07:04.900508 13651 raft_consensus.cc:348] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 152330aead374ba289c8b794390d7bbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } } peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } }
I20260812 19:07:04.900770 13651 raft_consensus.cc:374] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 152330aead374ba289c8b794390d7bbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:04.900828 13651 raft_consensus.cc:729] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 152330aead374ba289c8b794390d7bbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 152330aead374ba289c8b794390d7bbf, State: Initialized, Role: FOLLOWER
W20260812 19:07:04.900930 13313 tablet.cc:2369] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:07:04.901321 13651 consensus_queue.cc:260] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 152330aead374ba289c8b794390d7bbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } } peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } }
I20260812 19:07:04.902079 13475 heartbeater.cc:503] Master 127.25.130.62:46215 was elected leader, sending a full tablet report...
I20260812 19:07:04.902352 13651 ts_tablet_manager.cc:1428] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 152330aead374ba289c8b794390d7bbf: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260812 19:07:04.902653 13653 raft_consensus.cc:348] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 470846748a814ae1bfaf876b0a52b9f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } } peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } }
I20260812 19:07:04.902940 13653 raft_consensus.cc:374] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 470846748a814ae1bfaf876b0a52b9f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:04.902995 13653 raft_consensus.cc:729] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 470846748a814ae1bfaf876b0a52b9f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 470846748a814ae1bfaf876b0a52b9f0, State: Initialized, Role: FOLLOWER
I20260812 19:07:04.903481 13653 consensus_queue.cc:260] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 470846748a814ae1bfaf876b0a52b9f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } } peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } }
I20260812 19:07:04.904300 13635 heartbeater.cc:503] Master 127.25.130.62:46215 was elected leader, sending a full tablet report...
I20260812 19:07:04.904476 13653 ts_tablet_manager.cc:1428] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 470846748a814ae1bfaf876b0a52b9f0: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260812 19:07:05.073848 13636 tablet.cc:2369] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 470846748a814ae1bfaf876b0a52b9f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 19:07:05.109158 13476 tablet.cc:2369] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 152330aead374ba289c8b794390d7bbf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:07:05.172012 13321 raft_consensus.cc:482] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:05.172248 13321 raft_consensus.cc:504] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } } peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } }
I20260812 19:07:05.173434 13321 leader_election.cc:288] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 152330aead374ba289c8b794390d7bbf (127.25.130.2:38941), 470846748a814ae1bfaf876b0a52b9f0 (127.25.130.3:45501)
I20260812 19:07:05.178351 13648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:05.173941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:43735 (local address 127.25.130.3:45501)
0812 19:07:05.174076 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:05.174080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:05.174212 (+ 132us) server_negotiation.cc:407] Connection header received
0812 19:07:05.174488 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:05.174491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:05.174539 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:05.174604 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:05.174912 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:05.175709 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:05.177312 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:05.177763 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:05.177867 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:05.177924 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:05.178139 (+ 215us) server_negotiation.cc:299] Negotiation successful
0812 19:07:05.178218 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 19:07:05.178351 13646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:05.174099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:59367 (local address 127.25.130.2:38941)
0812 19:07:05.174223 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:05.174228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:05.174371 (+ 143us) server_negotiation.cc:407] Connection header received
0812 19:07:05.174752 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:05.174755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:05.174807 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:05.174874 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:05.175107 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:05.175593 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:05.177280 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:05.177791 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:05.177906 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:05.177957 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:05.178142 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 19:07:05.178219 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 19:07:05.178447 13658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:05.173720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:45501 (local address 127.25.130.1:43735)
0812 19:07:05.174120 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:05.174135 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:05.174314 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:05.174625 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:05.174628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:05.174646 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:05.174785 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:05.174792 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:05.175812 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:05.175815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:05.177150 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:05.177162 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:05.177889 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:05.177892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:05.177997 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:05.178041 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 19:07:05.178112 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 19:07:05.178272 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":63,"thread_start_us":126,"threads_started":1}
I20260812 19:07:05.178610 13659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:05.173721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:38941 (local address 127.25.130.1:59367)
0812 19:07:05.174275 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:05.174290 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:05.174394 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:05.174915 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:05.174918 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:05.174931 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:05.175005 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:05.175009 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:05.175699 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:05.175702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:05.177151 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:05.177162 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:05.177918 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:05.177922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:05.178025 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:05.178044 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 19:07:05.178110 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 19:07:05.178274 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":244,"thread_start_us":62,"threads_started":1}
I20260812 19:07:05.179193 13577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e30ca75b9e147e3ad8ac1c45b7f01f0" candidate_uuid: "eecd5f36e76c4532ab715479e18f7b76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "470846748a814ae1bfaf876b0a52b9f0" is_pre_election: true
I20260812 19:07:05.179276 13421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e30ca75b9e147e3ad8ac1c45b7f01f0" candidate_uuid: "eecd5f36e76c4532ab715479e18f7b76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "152330aead374ba289c8b794390d7bbf" is_pre_election: true
I20260812 19:07:05.179499 13577 raft_consensus.cc:2454] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 470846748a814ae1bfaf876b0a52b9f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eecd5f36e76c4532ab715479e18f7b76 in term 0.
I20260812 19:07:05.179561 13421 raft_consensus.cc:2454] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 152330aead374ba289c8b794390d7bbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eecd5f36e76c4532ab715479e18f7b76 in term 0.
I20260812 19:07:05.179992 13176 leader_election.cc:302] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 152330aead374ba289c8b794390d7bbf, 470846748a814ae1bfaf876b0a52b9f0, eecd5f36e76c4532ab715479e18f7b76; no voters:
I20260812 19:07:05.180262 13321 raft_consensus.cc:2781] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:07:05.180339 13321 raft_consensus.cc:482] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:05.180371 13321 raft_consensus.cc:3037] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:05.181466 13321 raft_consensus.cc:504] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } } peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } }
I20260812 19:07:05.181869 13321 leader_election.cc:288] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [CANDIDATE]: Term 1 election: Requested vote from peers 152330aead374ba289c8b794390d7bbf (127.25.130.2:38941), 470846748a814ae1bfaf876b0a52b9f0 (127.25.130.3:45501)
I20260812 19:07:05.182227 13577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e30ca75b9e147e3ad8ac1c45b7f01f0" candidate_uuid: "eecd5f36e76c4532ab715479e18f7b76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "470846748a814ae1bfaf876b0a52b9f0"
I20260812 19:07:05.182235 13421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e30ca75b9e147e3ad8ac1c45b7f01f0" candidate_uuid: "eecd5f36e76c4532ab715479e18f7b76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "152330aead374ba289c8b794390d7bbf"
I20260812 19:07:05.182343 13421 raft_consensus.cc:3037] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 152330aead374ba289c8b794390d7bbf [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:05.182375 13577 raft_consensus.cc:3037] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 470846748a814ae1bfaf876b0a52b9f0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:05.183427 13421 raft_consensus.cc:2454] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 152330aead374ba289c8b794390d7bbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecd5f36e76c4532ab715479e18f7b76 in term 1.
I20260812 19:07:05.183483 13577 raft_consensus.cc:2454] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 470846748a814ae1bfaf876b0a52b9f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecd5f36e76c4532ab715479e18f7b76 in term 1.
I20260812 19:07:05.183736 13175 leader_election.cc:302] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152330aead374ba289c8b794390d7bbf, eecd5f36e76c4532ab715479e18f7b76; no voters:
I20260812 19:07:05.183904 13321 raft_consensus.cc:2781] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:07:05.184110 13321 raft_consensus.cc:686] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [term 1 LEADER]: Becoming Leader. State: Replica: eecd5f36e76c4532ab715479e18f7b76, State: Running, Role: LEADER
I20260812 19:07:05.184367 13321 consensus_queue.cc:237] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } } peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } }
I20260812 19:07:05.187350 13090 catalog_manager.cc:5495] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 reported cstate change: term changed from 0 to 1, leader changed from <none> to eecd5f36e76c4532ab715479e18f7b76 (127.25.130.1). New cstate: current_term: 1 leader_uuid: "eecd5f36e76c4532ab715479e18f7b76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } health_report { overall_health: HEALTHY } } }
I20260812 19:07:05.243443 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:05.239754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55968 (local address 127.25.130.1:39005)
0812 19:07:05.240069 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:05.240076 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:05.240095 (+ 19us) server_negotiation.cc:407] Connection header received
0812 19:07:05.240172 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:05.240174 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:05.240229 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:05.240340 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:07:05.240579 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:05.241102 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:05.241954 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:05.242190 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:05.242281 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:05.242564 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:07:05.243017 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:07:05.243138 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:05.243203 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 19:07:05.243268 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":114,"threads_started":1}
I20260812 19:07:05.247610 13577 raft_consensus.cc:1261] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 470846748a814ae1bfaf876b0a52b9f0 [term 1 FOLLOWER]: Refusing update from remote peer eecd5f36e76c4532ab715479e18f7b76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:05.247610 13421 raft_consensus.cc:1261] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 152330aead374ba289c8b794390d7bbf [term 1 FOLLOWER]: Refusing update from remote peer eecd5f36e76c4532ab715479e18f7b76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:05.248353 13660 consensus_queue.cc:1035] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:07:05.248536 13321 consensus_queue.cc:1035] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:07:05.267135 13089 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:07:05.267310 13089 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:07:05.269901 13089 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875: Sending DeleteTablet for 3 replicas of tablet 9e30ca75b9e147e3ad8ac1c45b7f01f0
I20260812 19:07:05.270781 13401 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e30ca75b9e147e3ad8ac1c45b7f01f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:07:05 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:48524
I20260812 19:07:05.270787 13555 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e30ca75b9e147e3ad8ac1c45b7f01f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:07:05 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:50438
I20260812 19:07:05.271389 13680 tablet_replica.cc:330] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 470846748a814ae1bfaf876b0a52b9f0: stopping tablet replica
I20260812 19:07:05.271427 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 13132
I20260812 19:07:05.271449 13226 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e30ca75b9e147e3ad8ac1c45b7f01f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:07:05 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:55962
I20260812 19:07:05.271765 13679 tablet_replica.cc:330] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 152330aead374ba289c8b794390d7bbf: stopping tablet replica
I20260812 19:07:05.272164 13679 raft_consensus.cc:2229] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 152330aead374ba289c8b794390d7bbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:05.272452 13680 raft_consensus.cc:2229] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 470846748a814ae1bfaf876b0a52b9f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:05.272809 13680 raft_consensus.cc:2258] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 470846748a814ae1bfaf876b0a52b9f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:05.273123 13679 raft_consensus.cc:2258] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 152330aead374ba289c8b794390d7bbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:05.275552 13680 ts_tablet_manager.cc:1905] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 470846748a814ae1bfaf876b0a52b9f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:07:05.276247 13679 ts_tablet_manager.cc:1905] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 152330aead374ba289c8b794390d7bbf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:07:05.278573 13679 ts_tablet_manager.cc:1918] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 152330aead374ba289c8b794390d7bbf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:07:05.278676 13679 log.cc:1196] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 152330aead374ba289c8b794390d7bbf: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/wal/wals/9e30ca75b9e147e3ad8ac1c45b7f01f0
I20260812 19:07:05.278781 13680 ts_tablet_manager.cc:1918] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 470846748a814ae1bfaf876b0a52b9f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:07:05.278884 13680 log.cc:1196] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 470846748a814ae1bfaf876b0a52b9f0: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/wal/wals/9e30ca75b9e147e3ad8ac1c45b7f01f0
I20260812 19:07:05.279004 13679 ts_tablet_manager.cc:1939] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 152330aead374ba289c8b794390d7bbf: Deleting consensus metadata
I20260812 19:07:05.279240 13680 ts_tablet_manager.cc:1939] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P 470846748a814ae1bfaf876b0a52b9f0: Deleting consensus metadata
I20260812 19:07:05.280263 13072 catalog_manager.cc:4841] TS 152330aead374ba289c8b794390d7bbf (127.25.130.2:38941): tablet 9e30ca75b9e147e3ad8ac1c45b7f01f0 (table test-table [id=8a92b630debf445cb6c9c3d283e5b4fa]) successfully deleted
I20260812 19:07:05.280406 13073 catalog_manager.cc:4841] TS 470846748a814ae1bfaf876b0a52b9f0 (127.25.130.3:45501): tablet 9e30ca75b9e147e3ad8ac1c45b7f01f0 (table test-table [id=8a92b630debf445cb6c9c3d283e5b4fa]) successfully deleted
W20260812 19:07:05.283737 13072 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.130.1:39005 (error 108)
I20260812 19:07:05.284664 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 13318
I20260812 19:07:05.284890 13323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:05.284574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:39005 (local address 127.0.0.1:55972)
0812 19:07:05.284721 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:05.284800 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.1:39005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 19:07:05.287959 13072 catalog_manager.cc:4584] TS eecd5f36e76c4532ab715479e18f7b76 (127.25.130.1:39005): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9e30ca75b9e147e3ad8ac1c45b7f01f0: Network error: Client connection negotiation failed: client connection to 127.25.130.1:39005: connect: Connection refused (error 111)
I20260812 19:07:05.293566 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 13478
I20260812 19:07:05.305536 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 13052
I20260812 19:07:25.315092 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:46215
--webserver_interface=127.25.130.62
--webserver_port=38947
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:36571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:46215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:07:25.443845 13682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:25.444235 13682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:25.444301 13682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:25.448592 13682 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 19:07:25.448669 13682 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 19:07:25.448693 13682 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 19:07:25.448709 13682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:07:25.448733 13682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:25.448767 13682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:25.448797 13682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:07:25.448825 13682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:07:25.454748 13682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:36571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:46215
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:46215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=38947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:25.456211 13682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:25.457602 13682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:25.467882 13687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:25.467959 13682 server_base.cc:1034] running on GCE node
W20260812 19:07:25.467882 13689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:25.467882 13691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:25.468873 13682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:25.470388 13682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:25.471571 13682 hybrid_clock.cc:648] HybridClock initialized: now 1786561645471539 us; error 53 us; skew 500 ppm
Aug 12 19:07:25 dist-test-slave-49h3 krb5kdc[12405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561645, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:07:25.475164 13682 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:07:25.476320 13682 webserver.cc:460] Webserver started at http://127.25.130.62:38947/ using document root <none> and password file <none>
I20260812 19:07:25.476989 13682 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:25.477057 13682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:25.481680 13682 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 19:07:25.484105 13704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:25.485543 13682 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:07:25.485674 13682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "3f33a968c92f4a55b07f34cfee721875"
format_stamp: "Formatted at 2026-08-12 19:06:42 on dist-test-slave-49h3"
server_key: "5a77b2e4618563069ebf0b91b01060b2"
server_key_iv: "cdcf42f59f6fe7fda2175a43131c3361"
server_key_version: "encryptionkey@0"
I20260812 19:07:25.486330 13682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:25.501202 13682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:25.504667 13682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:25.504865 13682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:25.513381 13682 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:46215
I20260812 19:07:25.513726 13762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:46215 every 8 connection(s)
I20260812 19:07:25.514859 13682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:07:25.520397 13763 sys_catalog.cc:263] Verifying existing consensus state
I20260812 19:07:25.522529 13763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875: Bootstrap starting.
I20260812 19:07:25.524581 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 13682
I20260812 19:07:25.525092 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:39005
--local_ip_for_outbound_sockets=127.25.130.1
--tserver_master_addrs=127.25.130.62:46215
--webserver_port=40437
--webserver_interface=127.25.130.1
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:36571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:07:25.545650 13763 log.cc:824] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:25.559002 13763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:07:25.559522 13763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875: Bootstrap complete.
I20260812 19:07:25.564433 13763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f33a968c92f4a55b07f34cfee721875" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46215 } }
I20260812 19:07:25.564962 13763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f33a968c92f4a55b07f34cfee721875, State: Initialized, Role: FOLLOWER
I20260812 19:07:25.565667 13763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f33a968c92f4a55b07f34cfee721875" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46215 } }
I20260812 19:07:25.565825 13763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:07:25.565905 13763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:07:25.566028 13763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:25.568440 13763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f33a968c92f4a55b07f34cfee721875" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46215 } }
I20260812 19:07:25.568832 13763 leader_election.cc:302] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f33a968c92f4a55b07f34cfee721875; no voters:
I20260812 19:07:25.569262 13763 leader_election.cc:288] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 19:07:25.569393 13767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:25.569726 13767 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [term 3 LEADER]: Becoming Leader. State: Replica: 3f33a968c92f4a55b07f34cfee721875, State: Running, Role: LEADER
I20260812 19:07:25.570144 13767 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f33a968c92f4a55b07f34cfee721875" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46215 } }
I20260812 19:07:25.570830 13763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:07:25.571707 13768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "3f33a968c92f4a55b07f34cfee721875" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f33a968c92f4a55b07f34cfee721875" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46215 } } }
I20260812 19:07:25.571861 13768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [sys.catalog]: This master's current role is: LEADER
I20260812 19:07:25.572177 13769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f33a968c92f4a55b07f34cfee721875. Latest consensus state: current_term: 3 leader_uuid: "3f33a968c92f4a55b07f34cfee721875" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f33a968c92f4a55b07f34cfee721875" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 46215 } } }
I20260812 19:07:25.572271 13769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875 [sys.catalog]: This master's current role is: LEADER
I20260812 19:07:25.572847 13777 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:07:25.579025 13777 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:07:25.580080 13777 catalog_manager.cc:1259] Loaded cluster ID: 89d5521c97c94a16b0ca3583b2bccb36
I20260812 19:07:25.580144 13777 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:07:25.582818 13777 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:07:25.583603 13777 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875: Loaded TSK: 0
I20260812 19:07:25.583694 13777 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3f33a968c92f4a55b07f34cfee721875: Loaded TSK: 1
F20260812 19:07:25.584558 13777 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f92b77289ad google::LogMessage::Fail() at ??:0
@ 0x7f92b772b88a google::LogMessage::SendToLog() at ??:0
@ 0x7f92b77284de google::LogMessage::Flush() at ??:0
@ 0x7f92b7729a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f92ba3ec18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f92ba3e4f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f92c208d7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f92c206799c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f92c2068178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f92c20a9df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f92c20c882b std::function<>::operator()() at ??:0
@ 0x7f92c2067b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f92c2068f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f92c2065579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f92c20a9640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f92b997a3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f92b997ac83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f92b997c3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f92b996b512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f92bafaf6db start_thread at ??:0
@ 0x7f92b622661f clone at ??:0
W20260812 19:07:25.682189 13765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:25.682448 13765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:25.682513 13765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:25.686905 13765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:25.686975 13765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:25.687070 13765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:07:25.693022 13765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:36571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:39005
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=40437
--tserver_master_addrs=127.25.130.62:46215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:25.694559 13765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:25.695849 13765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:07:25.707230 13765 server_base.cc:1034] running on GCE node
W20260812 19:07:25.707396 13817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:25.707432 13814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:25.707463 13815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:25.707999 13765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:25.708777 13765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:25.709954 13765 hybrid_clock.cc:648] HybridClock initialized: now 1786561645709925 us; error 42 us; skew 500 ppm
Aug 12 19:07:25 dist-test-slave-49h3 krb5kdc[12405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561645, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:07:25.713099 13765 init.cc:377] Logged in from keytab as kudu/127.25.130.1@KRBTEST.COM (short username kudu)
I20260812 19:07:25.714176 13765 webserver.cc:460] Webserver started at http://127.25.130.1:40437/ using document root <none> and password file <none>
I20260812 19:07:25.714833 13765 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:25.714905 13765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:25.719763 13765 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 19:07:25.722533 13824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:25.724177 13765 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:07:25.724316 13765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "eecd5f36e76c4532ab715479e18f7b76"
format_stamp: "Formatted at 2026-08-12 19:06:42 on dist-test-slave-49h3"
server_key: "c0d79c6e412dcbd20589eff096ab95df"
server_key_iv: "85e22406d604e11de37086e02b06af69"
server_key_version: "encryptionkey@0"
I20260812 19:07:25.725035 13765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:25.760186 13765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:25.764098 13765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:25.764329 13765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:25.765138 13765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:07:25.766609 13834 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 19:07:25.773209 13765 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 19:07:25.773280 13765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260812 19:07:25.773368 13765 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 19:07:25.775503 13765 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 19:07:25.775553 13765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260812 19:07:25.775957 13834 tablet_bootstrap.cc:492] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76: Bootstrap starting.
I20260812 19:07:25.787532 13765 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:39005
I20260812 19:07:25.787827 13963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:39005 every 8 connection(s)
I20260812 19:07:25.788816 13765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:07:25.795914 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 13765
I20260812 19:07:25.796386 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:38941
--local_ip_for_outbound_sockets=127.25.130.2
--tserver_master_addrs=127.25.130.62:46215
--webserver_port=42143
--webserver_interface=127.25.130.2
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:36571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:07:25.804737 13834 log.cc:824] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:25.813252 13834 tablet_bootstrap.cc:492] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:07:25.813679 13834 tablet_bootstrap.cc:492] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76: Bootstrap complete.
I20260812 19:07:25.814829 13834 ts_tablet_manager.cc:1397] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76: Time spent bootstrapping tablet: real 0.039s user 0.018s sys 0.017s
I20260812 19:07:25.817660 13834 raft_consensus.cc:348] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } } peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } }
I20260812 19:07:25.818164 13834 raft_consensus.cc:729] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecd5f36e76c4532ab715479e18f7b76, State: Initialized, Role: FOLLOWER
I20260812 19:07:25.818722 13834 consensus_queue.cc:260] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } } peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } }
I20260812 19:07:25.819859 13834 ts_tablet_manager.cc:1428] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
W20260812 19:07:25.851577 13966 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:07:25.791285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.1:36781)
0812 19:07:25.791885 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:25.791948 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:25.792921 (+ 973us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:25.850561 (+ 57640us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":158,"threads_started":1}
W20260812 19:07:25.852363 13964 heartbeater.cc:644] Failed to heartbeat to 127.25.130.62:46215 (0 consecutive failures): Network error: Failed to ping master at 127.25.130.62:46215: Client connection negotiation failed: client connection to 127.25.130.62:46215: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 19:07:25.853358 13966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:25.853033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.1:40443)
0812 19:07:25.853194 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:25.853279 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 19:07:25.854176 13966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:25.853939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.1:52659)
0812 19:07:25.854067 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:25.854115 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 19:07:25.854359 13964 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 19:07:25.946873 13967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:25.947177 13967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:25.947222 13967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:25.951284 13967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:25.951344 13967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:25.951406 13967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:07:25.956775 13967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:36571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:38941
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=42143
--tserver_master_addrs=127.25.130.62:46215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:25.958073 13967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:25.959277 13967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:25.970160 13976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:25.970155 13975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:25.970368 13967 server_base.cc:1034] running on GCE node
W20260812 19:07:25.970568 13979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:25.971004 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:25.971688 13967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:25.972836 13967 hybrid_clock.cc:648] HybridClock initialized: now 1786561645972810 us; error 40 us; skew 500 ppm
Aug 12 19:07:25 dist-test-slave-49h3 krb5kdc[12405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561645, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:07:25.975786 13967 init.cc:377] Logged in from keytab as kudu/127.25.130.2@KRBTEST.COM (short username kudu)
I20260812 19:07:25.976727 13967 webserver.cc:460] Webserver started at http://127.25.130.2:42143/ using document root <none> and password file <none>
I20260812 19:07:25.977366 13967 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:25.977440 13967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:25.981765 13967 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 19:07:25.983970 13988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:25.985236 13967 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:07:25.985347 13967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "152330aead374ba289c8b794390d7bbf"
format_stamp: "Formatted at 2026-08-12 19:06:43 on dist-test-slave-49h3"
server_key: "20e74f6a54c593565ef9b29ed53eae9c"
server_key_iv: "c828bcbf0a52ac50d62bdf0461090a67"
server_key_version: "encryptionkey@0"
I20260812 19:07:25.985978 13967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:25.996984 13967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:25.999872 13967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:26.000052 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:26.000676 13967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:07:26.001572 13967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:07:26.001621 13967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:26.001673 13967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:07:26.001700 13967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:26.012588 13967 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:38941
I20260812 19:07:26.012660 14129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:38941 every 8 connection(s)
I20260812 19:07:26.013983 13967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:07:26.018200 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 13967
I20260812 19:07:26.018689 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:45501
--local_ip_for_outbound_sockets=127.25.130.3
--tserver_master_addrs=127.25.130.62:46215
--webserver_port=46133
--webserver_interface=127.25.130.3
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:36571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:07:26.018819 14132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:26.016790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.2:60601)
0812 19:07:26.017668 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:26.017806 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":145,"threads_started":1}
W20260812 19:07:26.019351 14130 heartbeater.cc:644] Failed to heartbeat to 127.25.130.62:46215 (0 consecutive failures): Network error: Failed to ping master at 127.25.130.62:46215: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
I20260812 19:07:26.020159 14132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:26.019931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.2:48311)
0812 19:07:26.020043 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:26.020095 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 19:07:26.020810 14132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:26.020612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.2:37453)
0812 19:07:26.020713 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:26.020757 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 19:07:26.020970 14130 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 19:07:26.150437 14133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:26.150805 14133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:26.150856 14133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:26.155056 14133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:26.155134 14133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:26.155200 14133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:07:26.161072 14133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:36571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:45501
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=46133
--tserver_master_addrs=127.25.130.62:46215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:26.162484 14133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:26.163748 14133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:26.175336 14140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:26.175470 14133 server_base.cc:1034] running on GCE node
W20260812 19:07:26.175336 14142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:26.175678 14139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:26.176164 14133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:26.176970 14133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:26.178174 14133 hybrid_clock.cc:648] HybridClock initialized: now 1786561646178150 us; error 41 us; skew 500 ppm
Aug 12 19:07:26 dist-test-slave-49h3 krb5kdc[12405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561646, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:07:26.181579 14133 init.cc:377] Logged in from keytab as kudu/127.25.130.3@KRBTEST.COM (short username kudu)
I20260812 19:07:26.182684 14133 webserver.cc:460] Webserver started at http://127.25.130.3:46133/ using document root <none> and password file <none>
I20260812 19:07:26.183398 14133 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:26.183481 14133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:26.188303 14133 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 19:07:26.190933 14150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:26.192468 14133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 19:07:26.192618 14133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "470846748a814ae1bfaf876b0a52b9f0"
format_stamp: "Formatted at 2026-08-12 19:06:43 on dist-test-slave-49h3"
server_key: "90aa3c078c3a419be516eaabb56cb3fa"
server_key_iv: "e4a1395f8fa20c1cf4ab6e828fafe557"
server_key_version: "encryptionkey@0"
I20260812 19:07:26.193313 14133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:26.209235 14133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:26.212831 14133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:26.213037 14133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:26.213812 14133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:07:26.214813 14133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:07:26.214867 14133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:26.214926 14133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:07:26.214955 14133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:26.224296 14133 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:45501
I20260812 19:07:26.224323 14284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:45501 every 8 connection(s)
I20260812 19:07:26.225518 14133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:07:26.228173 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 14133
I20260812 19:07:26.230002 14287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:26.228008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.3:38635)
0812 19:07:26.228868 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:26.229014 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":151,"threads_started":1}
W20260812 19:07:26.230501 14285 heartbeater.cc:644] Failed to heartbeat to 127.25.130.62:46215 (0 consecutive failures): Network error: Failed to ping master at 127.25.130.62:46215: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
I20260812 19:07:26.231083 14287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:26.230872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.3:45477)
0812 19:07:26.230975 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:26.231024 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 19:07:26.231621 14287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:26.231434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.3:55389)
0812 19:07:26.231525 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:26.231573 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 19:07:26.231770 14285 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 19:07:26.855583 14290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:26.855080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.1:59489)
0812 19:07:26.855372 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:26.855445 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":87,"threads_started":1}
I20260812 19:07:27.022190 14291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:27.021676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.2:42375)
0812 19:07:27.021960 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:27.022050 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":81,"threads_started":1}
I20260812 19:07:27.232743 14293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:27.232310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.3:34257)
0812 19:07:27.232569 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:27.232626 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":76,"threads_started":1}
I20260812 19:07:27.481002 14294 raft_consensus.cc:482] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:27.481225 14294 raft_consensus.cc:504] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } } peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } }
I20260812 19:07:27.482506 14294 leader_election.cc:288] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 152330aead374ba289c8b794390d7bbf (127.25.130.2:38941), 470846748a814ae1bfaf876b0a52b9f0 (127.25.130.3:45501)
Aug 12 19:07:27 dist-test-slave-49h3 krb5kdc[12405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561645, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for kudu/127.25.130.2@KRBTEST.COM
Aug 12 19:07:27 dist-test-slave-49h3 krb5kdc[12405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561645, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for kudu/127.25.130.3@KRBTEST.COM
I20260812 19:07:27.500416 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:27.482910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:35777 (local address 127.25.130.3:45501)
0812 19:07:27.483488 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:27.483508 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:27.483664 (+ 156us) server_negotiation.cc:407] Connection header received
0812 19:07:27.484462 (+ 798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:27.484471 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:27.484956 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:27.485302 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:27.487127 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:27.488717 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:27.490807 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:27.491579 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:27.491769 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:27.495308 (+ 3539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:27.495313 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:27.495326 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:07:27.495362 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:27.497305 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:27.498388 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:27.498394 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:27.498400 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:27.498560 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:27.498836 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:27.498839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:27.498841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:27.499239 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:27.499358 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:27.499945 (+ 587us) server_negotiation.cc:299] Negotiation successful
0812 19:07:27.500223 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":161,"threads_started":1}
I20260812 19:07:27.500409 14299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:27.482784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:45501 (local address 127.25.130.1:35777)
0812 19:07:27.483568 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:27.483579 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:27.483672 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:27.485432 (+ 1760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:27.485439 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:27.486093 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:07:27.486930 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:27.486954 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:27.488940 (+ 1986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:27.488944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:27.490666 (+ 1722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:27.490681 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:27.491726 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:27.491730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:27.492096 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:27.492829 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:07:27.492867 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:07:27.495108 (+ 2241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:07:27.497498 (+ 2390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:27.497506 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:27.497524 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:27.497943 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:27.498654 (+ 711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:27.498657 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:27.498659 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:27.498747 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:27.499344 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:07:27.499347 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:07:27.499623 (+ 276us) client_negotiation.cc:769] Sending connection context
0812 19:07:27.499794 (+ 171us) client_negotiation.cc:240] Negotiation successful
0812 19:07:27.500192 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"mutex_wait_us":788,"spinlock_wait_cycles":57216,"thread_start_us":65,"threads_started":1}
I20260812 19:07:27.500463 14297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:27.483091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:45577 (local address 127.25.130.2:38941)
0812 19:07:27.483382 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:27.483412 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:27.483462 (+ 50us) server_negotiation.cc:407] Connection header received
0812 19:07:27.484463 (+ 1001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:27.484472 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:27.484956 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:27.485302 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:27.487134 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:27.488717 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:27.490807 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:27.491579 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:27.491769 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:27.495171 (+ 3402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:27.495178 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:27.495198 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:07:27.495239 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:27.497312 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:27.498090 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:27.498096 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:27.498101 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:27.498245 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:27.498689 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:27.498693 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:27.498695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:27.499213 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:27.499347 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:27.499945 (+ 598us) server_negotiation.cc:299] Negotiation successful
0812 19:07:27.500267 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":79,"threads_started":1}
I20260812 19:07:27.501060 14295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:27.482782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:38941 (local address 127.25.130.1:45577)
0812 19:07:27.483052 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:27.483070 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:27.483190 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:27.485432 (+ 2242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:27.485439 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:27.486093 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:07:27.486930 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:27.486954 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:27.488940 (+ 1986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:27.488944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:27.490666 (+ 1722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:27.490681 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:27.491726 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:27.491730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:27.492086 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:27.492829 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:07:27.492857 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:07:27.494900 (+ 2043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:07:27.497514 (+ 2614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:27.497518 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:27.497538 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:27.497946 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:27.498362 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:27.498365 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:27.498367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:27.498573 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:27.499335 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:07:27.499342 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:07:27.499623 (+ 281us) client_negotiation.cc:769] Sending connection context
0812 19:07:27.499794 (+ 171us) client_negotiation.cc:240] Negotiation successful
0812 19:07:27.500270 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":518,"spinlock_wait_cycles":26624,"thread_start_us":78,"threads_started":1}
W20260812 19:07:27.503223 13826 leader_election.cc:341] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 152330aead374ba289c8b794390d7bbf (127.25.130.2:38941): Not found: Tablet not found: 9e30ca75b9e147e3ad8ac1c45b7f01f0
W20260812 19:07:27.503435 13827 leader_election.cc:341] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 470846748a814ae1bfaf876b0a52b9f0 (127.25.130.3:45501): Not found: Tablet not found: 9e30ca75b9e147e3ad8ac1c45b7f01f0
I20260812 19:07:27.503523 13827 leader_election.cc:302] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eecd5f36e76c4532ab715479e18f7b76; no voters: 152330aead374ba289c8b794390d7bbf, 470846748a814ae1bfaf876b0a52b9f0
I20260812 19:07:27.503813 14294 raft_consensus.cc:2726] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:27.856835 14295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:27.856490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.1:46197)
0812 19:07:27.856642 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:27.856738 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 19:07:28.023559 14300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:28.023068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.2:52081)
0812 19:07:28.023388 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:28.023455 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":111,"threads_started":1}
I20260812 19:07:28.233974 14301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:28.233540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.3:57167)
0812 19:07:28.233816 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:28.233875 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":97,"threads_started":1}
I20260812 19:07:28.858258 14303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:28.857739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.1:34585)
0812 19:07:28.858056 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:28.858123 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":103,"threads_started":1}
I20260812 19:07:29.024771 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:29.024304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.2:33987)
0812 19:07:29.024586 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:29.024644 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":92,"threads_started":1}
I20260812 19:07:29.235277 14305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:29.234772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.3:35563)
0812 19:07:29.235066 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:29.235134 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":92,"threads_started":1}
I20260812 19:07:29.543586 14306 raft_consensus.cc:482] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:29.543772 14306 raft_consensus.cc:504] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } } peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } }
I20260812 19:07:29.544320 14306 leader_election.cc:288] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 152330aead374ba289c8b794390d7bbf (127.25.130.2:38941), 470846748a814ae1bfaf876b0a52b9f0 (127.25.130.3:45501)
W20260812 19:07:29.545322 13826 leader_election.cc:341] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 152330aead374ba289c8b794390d7bbf (127.25.130.2:38941): Not found: Tablet not found: 9e30ca75b9e147e3ad8ac1c45b7f01f0
W20260812 19:07:29.545471 13827 leader_election.cc:341] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 470846748a814ae1bfaf876b0a52b9f0 (127.25.130.3:45501): Not found: Tablet not found: 9e30ca75b9e147e3ad8ac1c45b7f01f0
I20260812 19:07:29.545564 13827 leader_election.cc:302] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eecd5f36e76c4532ab715479e18f7b76; no voters: 152330aead374ba289c8b794390d7bbf, 470846748a814ae1bfaf876b0a52b9f0
I20260812 19:07:29.545751 14306 raft_consensus.cc:2726] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:29.859622 14308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:29.859119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.1:42091)
0812 19:07:29.859425 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:29.859490 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":102,"threads_started":1}
I20260812 19:07:30.026546 14309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:30.025909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.2:57483)
0812 19:07:30.026290 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:30.026405 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":120,"threads_started":1}
I20260812 19:07:30.236801 14310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:30.236275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.3:44925)
0812 19:07:30.236606 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:30.236679 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":108,"threads_started":1}
I20260812 19:07:30.861333 14311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:30.860734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.1:46425)
0812 19:07:30.861100 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:30.861166 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":97,"threads_started":1}
I20260812 19:07:31.028092 14312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:31.027546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.2:33159)
0812 19:07:31.027889 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:31.027962 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":103,"threads_started":1}
I20260812 19:07:31.238468 14313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:31.237879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.3:56253)
0812 19:07:31.238242 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:31.238319 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":108,"threads_started":1}
I20260812 19:07:31.862844 14314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:31.862278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.1:37449)
0812 19:07:31.862632 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:31.862700 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":133,"threads_started":1}
I20260812 19:07:32.029428 14318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:32.028931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.2:46191)
0812 19:07:32.029229 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:32.029302 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":94,"threads_started":1}
I20260812 19:07:32.240049 14319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:32.239479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.3:39763)
0812 19:07:32.239835 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:32.239913 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":105,"threads_started":1}
I20260812 19:07:32.272861 14320 raft_consensus.cc:482] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:32.273072 14320 raft_consensus.cc:504] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } } peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } }
I20260812 19:07:32.273720 14320 leader_election.cc:288] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 152330aead374ba289c8b794390d7bbf (127.25.130.2:38941), 470846748a814ae1bfaf876b0a52b9f0 (127.25.130.3:45501)
W20260812 19:07:32.274757 13827 leader_election.cc:341] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 470846748a814ae1bfaf876b0a52b9f0 (127.25.130.3:45501): Not found: Tablet not found: 9e30ca75b9e147e3ad8ac1c45b7f01f0
W20260812 19:07:32.274900 13826 leader_election.cc:341] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 152330aead374ba289c8b794390d7bbf (127.25.130.2:38941): Not found: Tablet not found: 9e30ca75b9e147e3ad8ac1c45b7f01f0
I20260812 19:07:32.274969 13826 leader_election.cc:302] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eecd5f36e76c4532ab715479e18f7b76; no voters: 152330aead374ba289c8b794390d7bbf, 470846748a814ae1bfaf876b0a52b9f0
I20260812 19:07:32.275139 14320 raft_consensus.cc:2726] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:32.864336 14321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:32.863753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.1:46291)
0812 19:07:32.864105 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:32.864172 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":112,"threads_started":1}
I20260812 19:07:33.030978 14323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:33.030387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.2:45225)
0812 19:07:33.030788 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:33.030854 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":126,"threads_started":1}
I20260812 19:07:33.241573 14324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:33.240917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.3:37521)
0812 19:07:33.241322 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:33.241405 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":96,"threads_started":1}
I20260812 19:07:33.865906 14325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:33.865319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.1:45629)
0812 19:07:33.865693 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:33.865764 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":129,"threads_started":1}
I20260812 19:07:34.032493 14326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:34.031921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.2:55769)
0812 19:07:34.032276 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:34.032356 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":130,"threads_started":1}
I20260812 19:07:34.197469 14327 raft_consensus.cc:482] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:34.197667 14327 raft_consensus.cc:504] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } } peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } }
I20260812 19:07:34.198223 14327 leader_election.cc:288] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 152330aead374ba289c8b794390d7bbf (127.25.130.2:38941), 470846748a814ae1bfaf876b0a52b9f0 (127.25.130.3:45501)
W20260812 19:07:34.199173 13827 leader_election.cc:341] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 470846748a814ae1bfaf876b0a52b9f0 (127.25.130.3:45501): Not found: Tablet not found: 9e30ca75b9e147e3ad8ac1c45b7f01f0
W20260812 19:07:34.199312 13826 leader_election.cc:341] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 152330aead374ba289c8b794390d7bbf (127.25.130.2:38941): Not found: Tablet not found: 9e30ca75b9e147e3ad8ac1c45b7f01f0
I20260812 19:07:34.199435 13826 leader_election.cc:302] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eecd5f36e76c4532ab715479e18f7b76; no voters: 152330aead374ba289c8b794390d7bbf, 470846748a814ae1bfaf876b0a52b9f0
I20260812 19:07:34.199625 14327 raft_consensus.cc:2726] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:34.243022 14330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:34.242473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.3:41597)
0812 19:07:34.242824 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:34.242891 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":122,"threads_started":1}
I20260812 19:07:34.867368 14331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:34.866825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.1:54289)
0812 19:07:34.867172 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:34.867234 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":132,"threads_started":1}
I20260812 19:07:35.033936 14332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:35.033391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.2:42729)
0812 19:07:35.033741 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:35.033806 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":132,"threads_started":1}
I20260812 19:07:35.244462 14335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:35.243939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.3:48863)
0812 19:07:35.244271 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:35.244338 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":103,"threads_started":1}
I20260812 19:07:35.869118 14336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:35.868587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.1:47425)
0812 19:07:35.868924 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:35.868985 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":107,"threads_started":1}
I20260812 19:07:36.035406 14337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:36.034879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.2:46035)
0812 19:07:36.035210 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:36.035274 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":102,"threads_started":1}
I20260812 19:07:36.246109 14338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:36.245495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.3:60953)
0812 19:07:36.245891 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:36.245963 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":139,"threads_started":1}
I20260812 19:07:36.870584 14339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:36.870034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.1:44839)
0812 19:07:36.870385 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:36.870450 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":114,"threads_started":1}
I20260812 19:07:37.036815 14340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:37.036319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.2:40053)
0812 19:07:37.036617 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:37.036684 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":99,"threads_started":1}
I20260812 19:07:37.247732 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:37.247136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.3:43473)
0812 19:07:37.247503 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:37.247582 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":102,"threads_started":1}
I20260812 19:07:37.872120 14343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:37.871591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.1:36229)
0812 19:07:37.871919 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:37.871987 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":109,"threads_started":1}
I20260812 19:07:38.038395 14344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:38.037836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.2:56823)
0812 19:07:38.038183 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:38.038257 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":107,"threads_started":1}
I20260812 19:07:38.249307 14345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:38.248699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.3:44069)
0812 19:07:38.249097 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:38.249174 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":161,"threads_started":1}
I20260812 19:07:38.618701 14346 raft_consensus.cc:482] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:38.618906 14346 raft_consensus.cc:504] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "152330aead374ba289c8b794390d7bbf" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 38941 } } peers { permanent_uuid: "470846748a814ae1bfaf876b0a52b9f0" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 45501 } } peers { permanent_uuid: "eecd5f36e76c4532ab715479e18f7b76" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 39005 } }
I20260812 19:07:38.619428 14346 leader_election.cc:288] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 152330aead374ba289c8b794390d7bbf (127.25.130.2:38941), 470846748a814ae1bfaf876b0a52b9f0 (127.25.130.3:45501)
W20260812 19:07:38.620394 13827 leader_election.cc:341] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 470846748a814ae1bfaf876b0a52b9f0 (127.25.130.3:45501): Not found: Tablet not found: 9e30ca75b9e147e3ad8ac1c45b7f01f0
W20260812 19:07:38.620527 13826 leader_election.cc:341] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 152330aead374ba289c8b794390d7bbf (127.25.130.2:38941): Not found: Tablet not found: 9e30ca75b9e147e3ad8ac1c45b7f01f0
I20260812 19:07:38.620620 13826 leader_election.cc:302] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eecd5f36e76c4532ab715479e18f7b76; no voters: 152330aead374ba289c8b794390d7bbf, 470846748a814ae1bfaf876b0a52b9f0
I20260812 19:07:38.620777 14346 raft_consensus.cc:2726] T 9e30ca75b9e147e3ad8ac1c45b7f01f0 P eecd5f36e76c4532ab715479e18f7b76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:38.873620 14347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:38.873054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.1:50265)
0812 19:07:38.873390 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:38.873484 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":118,"threads_started":1}
I20260812 19:07:39.039930 14348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:39.039382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.2:42003)
0812 19:07:39.039729 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:39.039800 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":128,"threads_started":1}
I20260812 19:07:39.250942 14349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:39.250305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.3:43325)
0812 19:07:39.250724 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:39.250802 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":138,"threads_started":1}
I20260812 19:07:39.875092 14350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:39.874537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.1:35219)
0812 19:07:39.874892 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:39.874959 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":128,"threads_started":1}
I20260812 19:07:40.041517 14351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:40.040917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.2:32823)
0812 19:07:40.041271 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:40.041339 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":146,"threads_started":1}
I20260812 19:07:40.252627 14352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:40.251994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.3:45925)
0812 19:07:40.252403 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:40.252490 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":140,"threads_started":1}
I20260812 19:07:40.876777 14353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:40.876222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.1:51389)
0812 19:07:40.876588 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:40.876661 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":114,"threads_started":1}
I20260812 19:07:41.042963 14354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:41.042497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.2:45673)
0812 19:07:41.042798 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:41.042853 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":101,"threads_started":1}
I20260812 19:07:41.239610 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 13765
I20260812 19:07:41.250152 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 13967
I20260812 19:07:41.254637 14356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:41.253995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:46215 (local address 127.25.130.3:60473)
0812 19:07:41.254456 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:41.254519 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.62:46215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":154,"threads_started":1}
I20260812 19:07:41.259261 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 14133
2026-08-12T19:07:41Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (60131 ms)
[----------] 30 tests from SecurityITest (245268 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:07:41 dist-test-slave-49h3 krb5kdc[14360](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:07:41 dist-test-slave-49h3 krb5kdc[14360](info): set up 2 sockets
Aug 12 19:07:41 dist-test-slave-49h3 krb5kdc[14360](info): commencing operation
krb5kdc: starting...
W20260812 19:07:42.338809 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:07:42 dist-test-slave-49h3 krb5kdc[14360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:07:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:07:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:07:42.472111 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:45381
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:39715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:45381
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:07:42.601712 14375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:42.602057 14375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:42.602121 14375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:42.606340 14375 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:07:42.606426 14375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:42.606452 14375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:42.606474 14375 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:07:42.606505 14375 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:07:42.612260 14375 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:39715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:45381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:45381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:42.613601 14375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:42.614851 14375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:42.624243 14381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:42.624297 14375 server_base.cc:1034] running on GCE node
W20260812 19:07:42.624238 14384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:42.624238 14380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:42.625124 14375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:42.626471 14375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:42.627647 14375 hybrid_clock.cc:648] HybridClock initialized: now 1786561662627630 us; error 37 us; skew 500 ppm
Aug 12 19:07:42 dist-test-slave-49h3 krb5kdc[14360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561662, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:07:42.634413 14375 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:07:42.635565 14375 webserver.cc:460] Webserver started at http://127.25.130.62:35451/ using document root <none> and password file <none>
I20260812 19:07:42.636229 14375 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:42.636307 14375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:42.636547 14375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:42.638947 14375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "577626c886304730a53032ddab4923c7"
format_stamp: "Formatted at 2026-08-12 19:07:42 on dist-test-slave-49h3"
server_key: "b568d806a0d423161302bc25402bf02a"
server_key_iv: "d4de1757d4b05b6317a7c4979f7b2f45"
server_key_version: "encryptionkey@0"
I20260812 19:07:42.639556 14375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "577626c886304730a53032ddab4923c7"
format_stamp: "Formatted at 2026-08-12 19:07:42 on dist-test-slave-49h3"
server_key: "b568d806a0d423161302bc25402bf02a"
server_key_iv: "d4de1757d4b05b6317a7c4979f7b2f45"
server_key_version: "encryptionkey@0"
I20260812 19:07:42.643770 14375 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:07:42.646517 14392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:42.647979 14375 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:07:42.648118 14375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "577626c886304730a53032ddab4923c7"
format_stamp: "Formatted at 2026-08-12 19:07:42 on dist-test-slave-49h3"
server_key: "b568d806a0d423161302bc25402bf02a"
server_key_iv: "d4de1757d4b05b6317a7c4979f7b2f45"
server_key_version: "encryptionkey@0"
I20260812 19:07:42.648268 14375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:42.662261 14375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:42.665274 14375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:42.665484 14375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:42.672952 14375 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:45381
I20260812 19:07:42.672972 14456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:45381 every 8 connection(s)
I20260812 19:07:42.674042 14375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:07:42.677248 14457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:42.680495 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 14375
I20260812 19:07:42.680611 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:07:42.680888 26120 external_mini_cluster.cc:1428] Setting key 9f42f22c8afe093c3928960f6a01da00
I20260812 19:07:42.683791 14457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 577626c886304730a53032ddab4923c7: Bootstrap starting.
I20260812 19:07:42.686848 14457 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 577626c886304730a53032ddab4923c7: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:42.687851 14457 log.cc:824] T 00000000000000000000000000000000 P 577626c886304730a53032ddab4923c7: Log is configured to *not* fsync() on all Append() calls
Aug 12 19:07:42 dist-test-slave-49h3 krb5kdc[14360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:07:42.690409 14457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 577626c886304730a53032ddab4923c7: No bootstrap required, opened a new log
I20260812 19:07:42.694273 14457 raft_consensus.cc:348] T 00000000000000000000000000000000 P 577626c886304730a53032ddab4923c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "577626c886304730a53032ddab4923c7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 45381 } }
I20260812 19:07:42.694545 14457 raft_consensus.cc:374] T 00000000000000000000000000000000 P 577626c886304730a53032ddab4923c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:42.694623 14457 raft_consensus.cc:729] T 00000000000000000000000000000000 P 577626c886304730a53032ddab4923c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 577626c886304730a53032ddab4923c7, State: Initialized, Role: FOLLOWER
I20260812 19:07:42.695115 14457 consensus_queue.cc:260] T 00000000000000000000000000000000 P 577626c886304730a53032ddab4923c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "577626c886304730a53032ddab4923c7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 45381 } }
I20260812 19:07:42.695264 14457 raft_consensus.cc:388] T 00000000000000000000000000000000 P 577626c886304730a53032ddab4923c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:07:42.695333 14457 raft_consensus.cc:482] T 00000000000000000000000000000000 P 577626c886304730a53032ddab4923c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:07:42.695429 14457 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 577626c886304730a53032ddab4923c7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:42.695758 14460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:42.682374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35310 (local address 127.25.130.62:45381)
0812 19:07:42.682823 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:42.682840 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:42.682881 (+ 41us) server_negotiation.cc:407] Connection header received
0812 19:07:42.683548 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:42.683567 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:42.683927 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:42.684282 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:42.685149 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:42.686245 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:42.687399 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:42.687888 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:42.688047 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:42.690849 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:42.690855 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:42.690871 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:07:42.690902 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:42.692638 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:42.693099 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:42.693104 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:42.693110 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:42.693246 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:42.693533 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:42.693536 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:42.693538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:42.693986 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:42.694206 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:42.694503 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 19:07:42.694853 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":133,"threads_started":1}
I20260812 19:07:42.696456 14457 raft_consensus.cc:504] T 00000000000000000000000000000000 P 577626c886304730a53032ddab4923c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "577626c886304730a53032ddab4923c7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 45381 } }
I20260812 19:07:42.696802 14457 leader_election.cc:302] T 00000000000000000000000000000000 P 577626c886304730a53032ddab4923c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 577626c886304730a53032ddab4923c7; no voters:
I20260812 19:07:42.697152 14457 leader_election.cc:288] T 00000000000000000000000000000000 P 577626c886304730a53032ddab4923c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:07:42.697259 14463 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 577626c886304730a53032ddab4923c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:07:42.697506 14463 raft_consensus.cc:686] T 00000000000000000000000000000000 P 577626c886304730a53032ddab4923c7 [term 1 LEADER]: Becoming Leader. State: Replica: 577626c886304730a53032ddab4923c7, State: Running, Role: LEADER
I20260812 19:07:42.697868 14463 consensus_queue.cc:237] T 00000000000000000000000000000000 P 577626c886304730a53032ddab4923c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "577626c886304730a53032ddab4923c7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 45381 } }
I20260812 19:07:42.698552 14457 sys_catalog.cc:564] T 00000000000000000000000000000000 P 577626c886304730a53032ddab4923c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:07:42.699190 14465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 577626c886304730a53032ddab4923c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 577626c886304730a53032ddab4923c7. Latest consensus state: current_term: 1 leader_uuid: "577626c886304730a53032ddab4923c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "577626c886304730a53032ddab4923c7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 45381 } } }
I20260812 19:07:42.699429 14465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 577626c886304730a53032ddab4923c7 [sys.catalog]: This master's current role is: LEADER
I20260812 19:07:42.699671 14464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 577626c886304730a53032ddab4923c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "577626c886304730a53032ddab4923c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "577626c886304730a53032ddab4923c7" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 45381 } } }
I20260812 19:07:42.699769 14464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 577626c886304730a53032ddab4923c7 [sys.catalog]: This master's current role is: LEADER
W20260812 19:07:42.703549 14481 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 577626c886304730a53032ddab4923c7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:07:42.703652 14481 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:07:42.703747 14472 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:07:42.704586 14472 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:07:42.711225 14472 catalog_manager.cc:1347] Generated new cluster ID: 55a4566f69894b75b06832bb243c4809
I20260812 19:07:42.711315 14472 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:07:42.730942 14472 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:07:42.731945 14472 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:07:42.741616 14472 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 577626c886304730a53032ddab4923c7: Generated new TSK 0
I20260812 19:07:42.742422 14472 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
I20260812 19:07:42.776059 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:0
--local_ip_for_outbound_sockets=127.25.130.1
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:45381
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:39715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:07:42.898303 14487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:42.898607 14487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:42.898672 14487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:42.902935 14487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:42.903014 14487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:42.903100 14487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:07:42.908748 14487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:39715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:45381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:42.909936 14487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:42.910971 14487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:07:42.921046 14487 server_base.cc:1034] running on GCE node
W20260812 19:07:42.921002 14493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:42.921002 14496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:42.921005 14494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:42.921905 14487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:42.922572 14487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:42.923738 14487 hybrid_clock.cc:648] HybridClock initialized: now 1786561662923713 us; error 40 us; skew 500 ppm
Aug 12 19:07:42 dist-test-slave-49h3 krb5kdc[14360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561662, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:07:42.926534 14487 init.cc:377] Logged in from keytab as kudu/127.25.130.1@KRBTEST.COM (short username kudu)
I20260812 19:07:42.927500 14487 webserver.cc:460] Webserver started at http://127.25.130.1:39357/ using document root <none> and password file <none>
I20260812 19:07:42.928076 14487 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:42.928172 14487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:42.928407 14487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:42.930658 14487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-0/data/instance:
uuid: "32e25f10587443fa8727e2306d8ee182"
format_stamp: "Formatted at 2026-08-12 19:07:42 on dist-test-slave-49h3"
server_key: "e9bde8aa7b83b8865aa6be0c9efd8d03"
server_key_iv: "346d81b46fde4568081613ad3412f6e8"
server_key_version: "encryptionkey@0"
I20260812 19:07:42.931275 14487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance:
uuid: "32e25f10587443fa8727e2306d8ee182"
format_stamp: "Formatted at 2026-08-12 19:07:42 on dist-test-slave-49h3"
server_key: "e9bde8aa7b83b8865aa6be0c9efd8d03"
server_key_iv: "346d81b46fde4568081613ad3412f6e8"
server_key_version: "encryptionkey@0"
I20260812 19:07:42.935322 14487 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:07:42.938071 14504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:42.939318 14487 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:07:42.939447 14487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "32e25f10587443fa8727e2306d8ee182"
format_stamp: "Formatted at 2026-08-12 19:07:42 on dist-test-slave-49h3"
server_key: "e9bde8aa7b83b8865aa6be0c9efd8d03"
server_key_iv: "346d81b46fde4568081613ad3412f6e8"
server_key_version: "encryptionkey@0"
I20260812 19:07:42.939589 14487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:42.954774 14487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:42.957712 14487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:42.957904 14487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:42.958528 14487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:07:42.959388 14487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:07:42.959439 14487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:42.959501 14487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:07:42.959529 14487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:42.968463 14487 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:38083
I20260812 19:07:42.968490 14641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:38083 every 8 connection(s)
I20260812 19:07:42.969380 14487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:07:42.973928 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 14487
I20260812 19:07:42.974046 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance
I20260812 19:07:42.974310 26120 external_mini_cluster.cc:1428] Setting key c397c28051a992ac708c9426b4d7a729
Aug 12 19:07:42 dist-test-slave-49h3 krb5kdc[14360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561662, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:07:42.983855 14460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:42.971260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:58187 (local address 127.25.130.62:45381)
0812 19:07:42.971385 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:42.971389 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:42.972226 (+ 837us) server_negotiation.cc:407] Connection header received
0812 19:07:42.973544 (+ 1318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:42.973548 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:42.973614 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:42.973693 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:42.975098 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:42.975698 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:42.976876 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:42.977196 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:42.977298 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:42.980260 (+ 2962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:42.980263 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:42.980265 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:07:42.980292 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:42.981716 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:42.982261 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:42.982264 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:42.982265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:42.982353 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:42.982691 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:42.982695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:42.982696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:42.982898 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:42.982988 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:42.983535 (+ 547us) server_negotiation.cc:299] Negotiation successful
0812 19:07:42.983683 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
WARNING: no policy specified for kudu/127.25.130.2@KRBTEST.COM; defaulting to no policy
I20260812 19:07:42.984769 14645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:42.971579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:45381 (local address 127.25.130.1:58187)
0812 19:07:42.972062 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:42.972116 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:42.973299 (+ 1183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:42.973891 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:42.973898 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:42.974307 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:07:42.974936 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:42.974955 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:42.975829 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:42.975832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:42.976748 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:42.976759 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:42.977339 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:42.977342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:42.977534 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:42.978176 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:07:42.978197 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:07:42.980064 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:07:42.981845 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:42.981852 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:42.981871 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:42.982155 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:42.982440 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:42.982443 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:42.982445 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:42.982600 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:42.983002 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:07:42.983010 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:07:42.983292 (+ 282us) client_negotiation.cc:769] Sending connection context
0812 19:07:42.983471 (+ 179us) client_negotiation.cc:240] Negotiation successful
0812 19:07:42.983847 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":124,"threads_started":1}
I20260812 19:07:42.986128 14642 heartbeater.cc:347] Connected to a master server at 127.25.130.62:45381
I20260812 19:07:42.986390 14642 heartbeater.cc:464] Registering TS with master...
I20260812 19:07:42.987001 14642 heartbeater.cc:511] Master 127.25.130.62:45381 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.2@KRBTEST.COM" created.
I20260812 19:07:42.988739 14413 ts_manager.cc:194] Registered new tserver with Master: 32e25f10587443fa8727e2306d8ee182 (127.25.130.1:38083)
I20260812 19:07:42.990003 14413 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.1@KRBTEST.COM'} at 127.25.130.1:58187
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
I20260812 19:07:43.003779 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:0
--local_ip_for_outbound_sockets=127.25.130.2
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:45381
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:39715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:07:43.117065 14649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:43.117331 14649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:43.117390 14649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:43.121479 14649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:43.121538 14649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:43.121608 14649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:07:43.126916 14649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:39715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:45381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:43.128055 14649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:43.129084 14649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:43.138597 14655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:43.138767 14649 server_base.cc:1034] running on GCE node
W20260812 19:07:43.138588 14658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:43.138588 14656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:43.139295 14649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:43.139956 14649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:43.141132 14649 hybrid_clock.cc:648] HybridClock initialized: now 1786561663141119 us; error 31 us; skew 500 ppm
Aug 12 19:07:43 dist-test-slave-49h3 krb5kdc[14360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561663, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:07:43.143981 14649 init.cc:377] Logged in from keytab as kudu/127.25.130.2@KRBTEST.COM (short username kudu)
I20260812 19:07:43.144925 14649 webserver.cc:460] Webserver started at http://127.25.130.2:33475/ using document root <none> and password file <none>
I20260812 19:07:43.145539 14649 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:43.145614 14649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:43.145829 14649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:43.147951 14649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-1/data/instance:
uuid: "489cbb1717784038821fd9f35b855b28"
format_stamp: "Formatted at 2026-08-12 19:07:43 on dist-test-slave-49h3"
server_key: "f399885289da2b37fdc8e5739a1da88f"
server_key_iv: "ddd8fe7519ee608cbee4166733a4abf0"
server_key_version: "encryptionkey@0"
I20260812 19:07:43.148519 14649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance:
uuid: "489cbb1717784038821fd9f35b855b28"
format_stamp: "Formatted at 2026-08-12 19:07:43 on dist-test-slave-49h3"
server_key: "f399885289da2b37fdc8e5739a1da88f"
server_key_iv: "ddd8fe7519ee608cbee4166733a4abf0"
server_key_version: "encryptionkey@0"
I20260812 19:07:43.152390 14649 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 19:07:43.154953 14667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:43.156198 14649 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 19:07:43.156320 14649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "489cbb1717784038821fd9f35b855b28"
format_stamp: "Formatted at 2026-08-12 19:07:43 on dist-test-slave-49h3"
server_key: "f399885289da2b37fdc8e5739a1da88f"
server_key_iv: "ddd8fe7519ee608cbee4166733a4abf0"
server_key_version: "encryptionkey@0"
I20260812 19:07:43.156427 14649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:43.167716 14649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:43.170631 14649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:43.170816 14649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:43.171422 14649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:07:43.172267 14649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:07:43.172315 14649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:43.172372 14649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:07:43.172401 14649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:43.180959 14649 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:40877
I20260812 19:07:43.180995 14800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:40877 every 8 connection(s)
I20260812 19:07:43.181885 14649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
Aug 12 19:07:43 dist-test-slave-49h3 krb5kdc[14360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561663, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:07:43.191082 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 14649
I20260812 19:07:43.191215 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance
I20260812 19:07:43.191545 26120 external_mini_cluster.cc:1428] Setting key d9b3a278a3f0011dd7e2cf59b03782a5
I20260812 19:07:43.195485 14460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:43.183627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:55025 (local address 127.25.130.62:45381)
0812 19:07:43.183798 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:43.183803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:43.184597 (+ 794us) server_negotiation.cc:407] Connection header received
0812 19:07:43.185884 (+ 1287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:43.185887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:43.185938 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:43.186025 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:43.187166 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.187678 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.188804 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.189032 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.189131 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:43.191773 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:43.191776 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:43.191778 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:07:43.191807 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:43.193173 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:43.193775 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:43.193778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:43.193779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:43.193860 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:43.194231 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:43.194233 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:43.194234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:43.194423 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:43.194499 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:43.195188 (+ 689us) server_negotiation.cc:299] Negotiation successful
0812 19:07:43.195345 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 19:07:43.196842 14805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:43.183994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:45381 (local address 127.25.130.2:55025)
0812 19:07:43.184445 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:43.184487 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:43.185692 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:43.186173 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:43.186179 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:43.186563 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:07:43.187048 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:43.187062 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.187776 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.187779 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:43.188661 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:43.188670 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.189184 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.189187 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:43.189335 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:43.189915 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:07:43.189931 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:07:43.191526 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:07:43.193291 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:43.193298 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:43.193316 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:43.193691 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:43.193939 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:43.193942 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:43.193944 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:43.194144 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:43.194515 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:07:43.194522 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:07:43.194864 (+ 342us) client_negotiation.cc:769] Sending connection context
0812 19:07:43.195668 (+ 804us) client_negotiation.cc:240] Negotiation successful
0812 19:07:43.196031 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":117,"threads_started":1}
I20260812 19:07:43.198268 14802 heartbeater.cc:347] Connected to a master server at 127.25.130.62:45381
I20260812 19:07:43.198539 14802 heartbeater.cc:464] Registering TS with master...
I20260812 19:07:43.199070 14802 heartbeater.cc:511] Master 127.25.130.62:45381 requested a full tablet report, sending...
I20260812 19:07:43.200260 14413 ts_manager.cc:194] Registered new tserver with Master: 489cbb1717784038821fd9f35b855b28 (127.25.130.2:40877)
I20260812 19:07:43.200832 14413 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.2@KRBTEST.COM'} at 127.25.130.2:55025
WARNING: no policy specified for kudu/127.25.130.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
I20260812 19:07:43.218559 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:0
--local_ip_for_outbound_sockets=127.25.130.3
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:45381
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:39715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:07:43.331614 14809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:43.331907 14809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:43.331970 14809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:43.336059 14809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:43.336122 14809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:43.336203 14809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:07:43.341562 14809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:39715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:45381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:43.342654 14809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:43.343710 14809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:43.353096 14817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:43.353245 14809 server_base.cc:1034] running on GCE node
W20260812 19:07:43.353091 14816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:43.353418 14820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:43.353827 14809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:43.354439 14809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:43.355593 14809 hybrid_clock.cc:648] HybridClock initialized: now 1786561663355572 us; error 34 us; skew 500 ppm
Aug 12 19:07:43 dist-test-slave-49h3 krb5kdc[14360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561663, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:07:43.358412 14809 init.cc:377] Logged in from keytab as kudu/127.25.130.3@KRBTEST.COM (short username kudu)
I20260812 19:07:43.359514 14809 webserver.cc:460] Webserver started at http://127.25.130.3:34823/ using document root <none> and password file <none>
I20260812 19:07:43.360204 14809 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:43.360306 14809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:43.360540 14809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:43.363194 14809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-2/data/instance:
uuid: "7bce241f6ed84e7a91016ce6826fa191"
format_stamp: "Formatted at 2026-08-12 19:07:43 on dist-test-slave-49h3"
server_key: "f44516f34e1247d7378c3a8ef43051fb"
server_key_iv: "a1b040e4d3dfeeedb0c62836b705da2c"
server_key_version: "encryptionkey@0"
I20260812 19:07:43.363889 14809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance:
uuid: "7bce241f6ed84e7a91016ce6826fa191"
format_stamp: "Formatted at 2026-08-12 19:07:43 on dist-test-slave-49h3"
server_key: "f44516f34e1247d7378c3a8ef43051fb"
server_key_iv: "a1b040e4d3dfeeedb0c62836b705da2c"
server_key_version: "encryptionkey@0"
I20260812 19:07:43.368484 14809 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 19:07:43.370949 14828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:43.372160 14809 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 19:07:43.372263 14809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "7bce241f6ed84e7a91016ce6826fa191"
format_stamp: "Formatted at 2026-08-12 19:07:43 on dist-test-slave-49h3"
server_key: "f44516f34e1247d7378c3a8ef43051fb"
server_key_iv: "a1b040e4d3dfeeedb0c62836b705da2c"
server_key_version: "encryptionkey@0"
I20260812 19:07:43.372355 14809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:43.390626 14809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:43.393783 14809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:43.393947 14809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:43.394520 14809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:07:43.395340 14809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:07:43.395388 14809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:43.395428 14809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:07:43.395449 14809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:43.404347 14809 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:40785
I20260812 19:07:43.404381 14967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:40785 every 8 connection(s)
I20260812 19:07:43.405247 14809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:07:43.406152 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 14809
I20260812 19:07:43.406291 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance
I20260812 19:07:43.406592 26120 external_mini_cluster.cc:1428] Setting key de6f3cd964386dfd1da610a4de1a7bd1
Aug 12 19:07:43 dist-test-slave-49h3 krb5kdc[14360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561663, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:07:43.418761 14460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:43.407481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:52967 (local address 127.25.130.62:45381)
0812 19:07:43.407615 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:43.407620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:43.408878 (+ 1258us) server_negotiation.cc:407] Connection header received
0812 19:07:43.410161 (+ 1283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:43.410164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:43.410211 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:43.410277 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:43.411396 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.411914 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.413017 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.413253 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.413332 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:43.415792 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:43.415794 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:43.415796 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:07:43.415822 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:43.416987 (+ 1165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:43.417432 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:43.417434 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:43.417435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:43.417540 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:43.417825 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:43.417828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:43.417829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:43.418006 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:43.418097 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:43.418523 (+ 426us) server_negotiation.cc:299] Negotiation successful
0812 19:07:43.418631 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 19:07:43.419646 14970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:43.408053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:45381 (local address 127.25.130.3:52967)
0812 19:07:43.408589 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:43.408781 (+ 192us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:43.409973 (+ 1192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:43.410414 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:43.410420 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:43.410796 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:07:43.411284 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:43.411298 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.412010 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.412012 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:43.412900 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:43.412909 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.413371 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.413374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:43.413566 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:43.414162 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:07:43.414181 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:07:43.415622 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:07:43.417085 (+ 1463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:43.417090 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:43.417103 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:43.417358 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:43.417612 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:43.417615 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:43.417616 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:43.417759 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:43.418075 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:07:43.418079 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:07:43.418317 (+ 238us) client_negotiation.cc:769] Sending connection context
0812 19:07:43.418469 (+ 152us) client_negotiation.cc:240] Negotiation successful
0812 19:07:43.418738 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":130,"threads_started":1}
I20260812 19:07:43.420828 14968 heartbeater.cc:347] Connected to a master server at 127.25.130.62:45381
I20260812 19:07:43.421051 14968 heartbeater.cc:464] Registering TS with master...
I20260812 19:07:43.421622 14968 heartbeater.cc:511] Master 127.25.130.62:45381 requested a full tablet report, sending...
I20260812 19:07:43.422618 14412 ts_manager.cc:194] Registered new tserver with Master: 7bce241f6ed84e7a91016ce6826fa191 (127.25.130.3:40785)
I20260812 19:07:43.423188 14412 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.3@KRBTEST.COM'} at 127.25.130.3:52967
I20260812 19:07:43.432549 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:07:43.442010 14460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:43.434739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35320 (local address 127.25.130.62:45381)
0812 19:07:43.434879 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:43.434884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:43.435016 (+ 132us) server_negotiation.cc:407] Connection header received
0812 19:07:43.435149 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:43.435151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:43.435195 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:43.435255 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:43.435976 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.436499 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.437371 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.437648 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.437740 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:43.439135 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:43.439138 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:43.439140 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:07:43.439167 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:43.440325 (+ 1158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:43.440739 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:43.440743 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:43.440744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:43.440827 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:43.441081 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:43.441084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:43.441086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:43.441318 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:43.441473 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:43.441724 (+ 251us) server_negotiation.cc:299] Negotiation successful
0812 19:07:43.441849 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 19:07:43.445134 14412 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35320:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:07:43.448225 14412 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:07:43.463459 14986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:43.457835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:38083 (local address 127.0.0.1:57184)
0812 19:07:43.458457 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:43.458468 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:43.458544 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:43.459004 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:43.459007 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:43.459026 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:43.459121 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:43.459126 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.460641 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.460644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:43.461866 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:43.461876 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.462880 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.462883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:43.462996 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:43.463046 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 19:07:43.463152 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 19:07:43.463288 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"mutex_wait_us":138,"thread_start_us":114,"threads_started":1}
I20260812 19:07:43.463547 14985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:43.457160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52458 (local address 127.25.130.2:40877)
0812 19:07:43.457609 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:43.457621 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:43.458261 (+ 640us) server_negotiation.cc:407] Connection header received
0812 19:07:43.458452 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:43.458458 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:43.458633 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:43.458757 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:43.459084 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.460464 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.462366 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.462974 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.463094 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:43.463235 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:43.463323 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 19:07:43.463418 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":110,"threads_started":1}
I20260812 19:07:43.463728 14984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:43.457150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:40785 (local address 127.0.0.1:58230)
0812 19:07:43.458155 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:43.458168 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:43.458257 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:43.459444 (+ 1187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:43.459446 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:43.459464 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:43.459561 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:43.459568 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.460602 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.460606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:43.462162 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:43.462174 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.463380 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.463383 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:43.463468 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:43.463489 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 19:07:43.463552 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 19:07:43.463615 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":947,"mutex_wait_us":50,"thread_start_us":103,"threads_started":1}
I20260812 19:07:43.464303 14987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:43.457896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58230 (local address 127.25.130.3:40785)
0812 19:07:43.459046 (+ 1150us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:43.459058 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:43.459078 (+ 20us) server_negotiation.cc:407] Connection header received
0812 19:07:43.459146 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:43.459150 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:43.459322 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:43.459449 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:43.459664 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.460451 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.462291 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.463080 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.463191 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:43.463322 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:43.464096 (+ 774us) server_negotiation.cc:299] Negotiation successful
0812 19:07:43.464180 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1038,"thread_start_us":175,"threads_started":1}
I20260812 19:07:43.464299 14988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:43.458046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57184 (local address 127.25.130.1:38083)
0812 19:07:43.458480 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:43.458498 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:43.458520 (+ 22us) server_negotiation.cc:407] Connection header received
0812 19:07:43.458664 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:43.458669 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:43.458849 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:43.459001 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:43.459214 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.460399 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.461990 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.462749 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.462861 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:43.463022 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:43.464050 (+ 1028us) server_negotiation.cc:299] Negotiation successful
0812 19:07:43.464157 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":149,"threads_started":1}
I20260812 19:07:43.463514 14983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:43.457002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:40877 (local address 127.0.0.1:52458)
0812 19:07:43.457553 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:43.457606 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:43.457759 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:43.458770 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:43.458776 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:43.458800 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:43.458951 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:43.458958 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.460618 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.460621 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:43.462248 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:43.462257 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.463108 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.463112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:43.463201 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:43.463224 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 19:07:43.463294 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 19:07:43.463370 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":108,"thread_start_us":110,"threads_started":1}
I20260812 19:07:43.467131 14565 tablet_service.cc:1467] Processing CreateTablet for tablet b64d63fe7f20455f8de68c3aade06d11 (DEFAULT_TABLE table=test-table [id=156ed44d41864f4b9fdf82cf9a13118e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:07:43.467303 14885 tablet_service.cc:1467] Processing CreateTablet for tablet b64d63fe7f20455f8de68c3aade06d11 (DEFAULT_TABLE table=test-table [id=156ed44d41864f4b9fdf82cf9a13118e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:07:43.467303 14726 tablet_service.cc:1467] Processing CreateTablet for tablet b64d63fe7f20455f8de68c3aade06d11 (DEFAULT_TABLE table=test-table [id=156ed44d41864f4b9fdf82cf9a13118e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:07:43.468364 14565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b64d63fe7f20455f8de68c3aade06d11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:43.468395 14726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b64d63fe7f20455f8de68c3aade06d11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:43.468413 14885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b64d63fe7f20455f8de68c3aade06d11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:43.475236 14989 tablet_bootstrap.cc:492] T b64d63fe7f20455f8de68c3aade06d11 P 32e25f10587443fa8727e2306d8ee182: Bootstrap starting.
I20260812 19:07:43.476739 14991 tablet_bootstrap.cc:492] T b64d63fe7f20455f8de68c3aade06d11 P 489cbb1717784038821fd9f35b855b28: Bootstrap starting.
I20260812 19:07:43.477838 14989 tablet_bootstrap.cc:654] T b64d63fe7f20455f8de68c3aade06d11 P 32e25f10587443fa8727e2306d8ee182: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:43.477818 14990 tablet_bootstrap.cc:492] T b64d63fe7f20455f8de68c3aade06d11 P 7bce241f6ed84e7a91016ce6826fa191: Bootstrap starting.
I20260812 19:07:43.478972 14991 tablet_bootstrap.cc:654] T b64d63fe7f20455f8de68c3aade06d11 P 489cbb1717784038821fd9f35b855b28: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:43.479162 14989 log.cc:824] T b64d63fe7f20455f8de68c3aade06d11 P 32e25f10587443fa8727e2306d8ee182: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:43.479758 14991 log.cc:824] T b64d63fe7f20455f8de68c3aade06d11 P 489cbb1717784038821fd9f35b855b28: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:43.479885 14990 tablet_bootstrap.cc:654] T b64d63fe7f20455f8de68c3aade06d11 P 7bce241f6ed84e7a91016ce6826fa191: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:43.481024 14990 log.cc:824] T b64d63fe7f20455f8de68c3aade06d11 P 7bce241f6ed84e7a91016ce6826fa191: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:43.482060 14991 tablet_bootstrap.cc:492] T b64d63fe7f20455f8de68c3aade06d11 P 489cbb1717784038821fd9f35b855b28: No bootstrap required, opened a new log
I20260812 19:07:43.482275 14991 ts_tablet_manager.cc:1397] T b64d63fe7f20455f8de68c3aade06d11 P 489cbb1717784038821fd9f35b855b28: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 19:07:43.482332 14989 tablet_bootstrap.cc:492] T b64d63fe7f20455f8de68c3aade06d11 P 32e25f10587443fa8727e2306d8ee182: No bootstrap required, opened a new log
I20260812 19:07:43.482532 14989 ts_tablet_manager.cc:1397] T b64d63fe7f20455f8de68c3aade06d11 P 32e25f10587443fa8727e2306d8ee182: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 19:07:43.482710 14990 tablet_bootstrap.cc:492] T b64d63fe7f20455f8de68c3aade06d11 P 7bce241f6ed84e7a91016ce6826fa191: No bootstrap required, opened a new log
I20260812 19:07:43.482877 14990 ts_tablet_manager.cc:1397] T b64d63fe7f20455f8de68c3aade06d11 P 7bce241f6ed84e7a91016ce6826fa191: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 19:07:43.485675 14990 raft_consensus.cc:348] T b64d63fe7f20455f8de68c3aade06d11 P 7bce241f6ed84e7a91016ce6826fa191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489cbb1717784038821fd9f35b855b28" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 40877 } } peers { permanent_uuid: "7bce241f6ed84e7a91016ce6826fa191" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40785 } } peers { permanent_uuid: "32e25f10587443fa8727e2306d8ee182" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 38083 } }
I20260812 19:07:43.485913 14990 raft_consensus.cc:374] T b64d63fe7f20455f8de68c3aade06d11 P 7bce241f6ed84e7a91016ce6826fa191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:43.485973 14990 raft_consensus.cc:729] T b64d63fe7f20455f8de68c3aade06d11 P 7bce241f6ed84e7a91016ce6826fa191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bce241f6ed84e7a91016ce6826fa191, State: Initialized, Role: FOLLOWER
I20260812 19:07:43.486251 14991 raft_consensus.cc:348] T b64d63fe7f20455f8de68c3aade06d11 P 489cbb1717784038821fd9f35b855b28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489cbb1717784038821fd9f35b855b28" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 40877 } } peers { permanent_uuid: "7bce241f6ed84e7a91016ce6826fa191" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40785 } } peers { permanent_uuid: "32e25f10587443fa8727e2306d8ee182" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 38083 } }
I20260812 19:07:43.486434 14990 consensus_queue.cc:260] T b64d63fe7f20455f8de68c3aade06d11 P 7bce241f6ed84e7a91016ce6826fa191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489cbb1717784038821fd9f35b855b28" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 40877 } } peers { permanent_uuid: "7bce241f6ed84e7a91016ce6826fa191" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40785 } } peers { permanent_uuid: "32e25f10587443fa8727e2306d8ee182" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 38083 } }
I20260812 19:07:43.486527 14991 raft_consensus.cc:374] T b64d63fe7f20455f8de68c3aade06d11 P 489cbb1717784038821fd9f35b855b28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:43.486585 14991 raft_consensus.cc:729] T b64d63fe7f20455f8de68c3aade06d11 P 489cbb1717784038821fd9f35b855b28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 489cbb1717784038821fd9f35b855b28, State: Initialized, Role: FOLLOWER
I20260812 19:07:43.487069 14968 heartbeater.cc:503] Master 127.25.130.62:45381 was elected leader, sending a full tablet report...
I20260812 19:07:43.487021 14991 consensus_queue.cc:260] T b64d63fe7f20455f8de68c3aade06d11 P 489cbb1717784038821fd9f35b855b28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489cbb1717784038821fd9f35b855b28" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 40877 } } peers { permanent_uuid: "7bce241f6ed84e7a91016ce6826fa191" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40785 } } peers { permanent_uuid: "32e25f10587443fa8727e2306d8ee182" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 38083 } }
I20260812 19:07:43.486251 14989 raft_consensus.cc:348] T b64d63fe7f20455f8de68c3aade06d11 P 32e25f10587443fa8727e2306d8ee182 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489cbb1717784038821fd9f35b855b28" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 40877 } } peers { permanent_uuid: "7bce241f6ed84e7a91016ce6826fa191" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40785 } } peers { permanent_uuid: "32e25f10587443fa8727e2306d8ee182" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 38083 } }
I20260812 19:07:43.487438 14989 raft_consensus.cc:374] T b64d63fe7f20455f8de68c3aade06d11 P 32e25f10587443fa8727e2306d8ee182 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:43.487521 14989 raft_consensus.cc:729] T b64d63fe7f20455f8de68c3aade06d11 P 32e25f10587443fa8727e2306d8ee182 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32e25f10587443fa8727e2306d8ee182, State: Initialized, Role: FOLLOWER
I20260812 19:07:43.487917 14989 consensus_queue.cc:260] T b64d63fe7f20455f8de68c3aade06d11 P 32e25f10587443fa8727e2306d8ee182 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489cbb1717784038821fd9f35b855b28" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 40877 } } peers { permanent_uuid: "7bce241f6ed84e7a91016ce6826fa191" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40785 } } peers { permanent_uuid: "32e25f10587443fa8727e2306d8ee182" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 38083 } }
I20260812 19:07:43.488057 14991 ts_tablet_manager.cc:1428] T b64d63fe7f20455f8de68c3aade06d11 P 489cbb1717784038821fd9f35b855b28: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 19:07:43.488077 14802 heartbeater.cc:503] Master 127.25.130.62:45381 was elected leader, sending a full tablet report...
I20260812 19:07:43.488816 14989 ts_tablet_manager.cc:1428] T b64d63fe7f20455f8de68c3aade06d11 P 32e25f10587443fa8727e2306d8ee182: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 19:07:43.489557 14990 ts_tablet_manager.cc:1428] T b64d63fe7f20455f8de68c3aade06d11 P 7bce241f6ed84e7a91016ce6826fa191: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 19:07:43.489646 14642 heartbeater.cc:503] Master 127.25.130.62:45381 was elected leader, sending a full tablet report...
I20260812 19:07:43.532407 14999 raft_consensus.cc:482] T b64d63fe7f20455f8de68c3aade06d11 P 32e25f10587443fa8727e2306d8ee182 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:43.532635 14999 raft_consensus.cc:504] T b64d63fe7f20455f8de68c3aade06d11 P 32e25f10587443fa8727e2306d8ee182 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489cbb1717784038821fd9f35b855b28" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 40877 } } peers { permanent_uuid: "7bce241f6ed84e7a91016ce6826fa191" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40785 } } peers { permanent_uuid: "32e25f10587443fa8727e2306d8ee182" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 38083 } }
I20260812 19:07:43.533981 14999 leader_election.cc:288] T b64d63fe7f20455f8de68c3aade06d11 P 32e25f10587443fa8727e2306d8ee182 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 489cbb1717784038821fd9f35b855b28 (127.25.130.2:40877), 7bce241f6ed84e7a91016ce6826fa191 (127.25.130.3:40785)
I20260812 19:07:43.538931 14985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:43.534511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:49803 (local address 127.25.130.2:40877)
0812 19:07:43.534651 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:43.534656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:43.534826 (+ 170us) server_negotiation.cc:407] Connection header received
0812 19:07:43.534973 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:43.534975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:43.535032 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:43.535097 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:43.535391 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.536103 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.537724 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.538323 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.538407 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:43.538450 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:43.538741 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 19:07:43.538816 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 19:07:43.538949 15001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:43.534240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:40877 (local address 127.25.130.1:49803)
0812 19:07:43.534717 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:43.534731 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:43.534857 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:43.535126 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:43.535129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:43.535144 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:43.535289 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:43.535297 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.536226 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.536229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:43.537565 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:43.537576 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.538532 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.538535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:43.538625 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:43.538644 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 19:07:43.538704 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 19:07:43.538783 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"mutex_wait_us":61,"thread_start_us":153,"threads_started":1}
I20260812 19:07:43.539114 14987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:43.534355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:54809 (local address 127.25.130.3:40785)
0812 19:07:43.534493 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:43.534498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:43.534934 (+ 436us) server_negotiation.cc:407] Connection header received
0812 19:07:43.535137 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:43.535140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:43.535189 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:43.535261 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:43.535555 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.536150 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.537734 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.538355 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.538479 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:43.538531 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:43.538877 (+ 346us) server_negotiation.cc:299] Negotiation successful
0812 19:07:43.538968 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 19:07:43.539559 15002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:43.534240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:40785 (local address 127.25.130.1:54809)
0812 19:07:43.534837 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:43.534852 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:43.535011 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:43.535290 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:43.535293 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:43.535310 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:43.535439 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:43.535445 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.536271 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.536275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:43.537584 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:43.537598 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.538598 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.538602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:43.538716 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:43.538744 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 19:07:43.539326 (+ 582us) client_negotiation.cc:240] Negotiation successful
0812 19:07:43.539414 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"mutex_wait_us":224,"thread_start_us":104,"threads_started":1}
I20260812 19:07:43.539934 14749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b64d63fe7f20455f8de68c3aade06d11" candidate_uuid: "32e25f10587443fa8727e2306d8ee182" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "489cbb1717784038821fd9f35b855b28" is_pre_election: true
I20260812 19:07:43.540269 14749 raft_consensus.cc:2454] T b64d63fe7f20455f8de68c3aade06d11 P 489cbb1717784038821fd9f35b855b28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32e25f10587443fa8727e2306d8ee182 in term 0.
I20260812 19:07:43.540314 14910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b64d63fe7f20455f8de68c3aade06d11" candidate_uuid: "32e25f10587443fa8727e2306d8ee182" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bce241f6ed84e7a91016ce6826fa191" is_pre_election: true
I20260812 19:07:43.540597 14910 raft_consensus.cc:2454] T b64d63fe7f20455f8de68c3aade06d11 P 7bce241f6ed84e7a91016ce6826fa191 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32e25f10587443fa8727e2306d8ee182 in term 0.
I20260812 19:07:43.540778 14506 leader_election.cc:302] T b64d63fe7f20455f8de68c3aade06d11 P 32e25f10587443fa8727e2306d8ee182 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32e25f10587443fa8727e2306d8ee182, 489cbb1717784038821fd9f35b855b28; no voters:
I20260812 19:07:43.541080 14999 raft_consensus.cc:2781] T b64d63fe7f20455f8de68c3aade06d11 P 32e25f10587443fa8727e2306d8ee182 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:07:43.541175 14999 raft_consensus.cc:482] T b64d63fe7f20455f8de68c3aade06d11 P 32e25f10587443fa8727e2306d8ee182 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:43.541225 14999 raft_consensus.cc:3037] T b64d63fe7f20455f8de68c3aade06d11 P 32e25f10587443fa8727e2306d8ee182 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:43.542361 14999 raft_consensus.cc:504] T b64d63fe7f20455f8de68c3aade06d11 P 32e25f10587443fa8727e2306d8ee182 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489cbb1717784038821fd9f35b855b28" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 40877 } } peers { permanent_uuid: "7bce241f6ed84e7a91016ce6826fa191" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40785 } } peers { permanent_uuid: "32e25f10587443fa8727e2306d8ee182" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 38083 } }
I20260812 19:07:43.542780 14999 leader_election.cc:288] T b64d63fe7f20455f8de68c3aade06d11 P 32e25f10587443fa8727e2306d8ee182 [CANDIDATE]: Term 1 election: Requested vote from peers 489cbb1717784038821fd9f35b855b28 (127.25.130.2:40877), 7bce241f6ed84e7a91016ce6826fa191 (127.25.130.3:40785)
I20260812 19:07:43.543215 14749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b64d63fe7f20455f8de68c3aade06d11" candidate_uuid: "32e25f10587443fa8727e2306d8ee182" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "489cbb1717784038821fd9f35b855b28"
I20260812 19:07:43.543215 14910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b64d63fe7f20455f8de68c3aade06d11" candidate_uuid: "32e25f10587443fa8727e2306d8ee182" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bce241f6ed84e7a91016ce6826fa191"
I20260812 19:07:43.543354 14749 raft_consensus.cc:3037] T b64d63fe7f20455f8de68c3aade06d11 P 489cbb1717784038821fd9f35b855b28 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:43.543352 14910 raft_consensus.cc:3037] T b64d63fe7f20455f8de68c3aade06d11 P 7bce241f6ed84e7a91016ce6826fa191 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:43.544718 14910 raft_consensus.cc:2454] T b64d63fe7f20455f8de68c3aade06d11 P 7bce241f6ed84e7a91016ce6826fa191 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32e25f10587443fa8727e2306d8ee182 in term 1.
I20260812 19:07:43.544718 14749 raft_consensus.cc:2454] T b64d63fe7f20455f8de68c3aade06d11 P 489cbb1717784038821fd9f35b855b28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32e25f10587443fa8727e2306d8ee182 in term 1.
I20260812 19:07:43.545051 14506 leader_election.cc:302] T b64d63fe7f20455f8de68c3aade06d11 P 32e25f10587443fa8727e2306d8ee182 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32e25f10587443fa8727e2306d8ee182, 489cbb1717784038821fd9f35b855b28; no voters:
I20260812 19:07:43.545230 14999 raft_consensus.cc:2781] T b64d63fe7f20455f8de68c3aade06d11 P 32e25f10587443fa8727e2306d8ee182 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:07:43.545507 14999 raft_consensus.cc:686] T b64d63fe7f20455f8de68c3aade06d11 P 32e25f10587443fa8727e2306d8ee182 [term 1 LEADER]: Becoming Leader. State: Replica: 32e25f10587443fa8727e2306d8ee182, State: Running, Role: LEADER
I20260812 19:07:43.545852 14999 consensus_queue.cc:237] T b64d63fe7f20455f8de68c3aade06d11 P 32e25f10587443fa8727e2306d8ee182 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489cbb1717784038821fd9f35b855b28" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 40877 } } peers { permanent_uuid: "7bce241f6ed84e7a91016ce6826fa191" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40785 } } peers { permanent_uuid: "32e25f10587443fa8727e2306d8ee182" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 38083 } }
I20260812 19:07:43.549369 14413 catalog_manager.cc:5495] T b64d63fe7f20455f8de68c3aade06d11 P 32e25f10587443fa8727e2306d8ee182 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32e25f10587443fa8727e2306d8ee182 (127.25.130.1). New cstate: current_term: 1 leader_uuid: "32e25f10587443fa8727e2306d8ee182" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489cbb1717784038821fd9f35b855b28" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 40877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bce241f6ed84e7a91016ce6826fa191" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32e25f10587443fa8727e2306d8ee182" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 38083 } health_report { overall_health: HEALTHY } } }
I20260812 19:07:43.562088 14988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:43.558684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57200 (local address 127.25.130.1:38083)
0812 19:07:43.558818 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:43.558821 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:43.558837 (+ 16us) server_negotiation.cc:407] Connection header received
0812 19:07:43.558934 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:43.558937 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:43.558994 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:43.559095 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:07:43.559289 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.559800 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.560702 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.560930 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.561006 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:43.561264 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:07:43.561745 (+ 481us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:07:43.561835 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:43.561889 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 19:07:43.561944 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 19:07:43.568305 14749 raft_consensus.cc:1261] T b64d63fe7f20455f8de68c3aade06d11 P 489cbb1717784038821fd9f35b855b28 [term 1 FOLLOWER]: Refusing update from remote peer 32e25f10587443fa8727e2306d8ee182: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:43.568305 14910 raft_consensus.cc:1261] T b64d63fe7f20455f8de68c3aade06d11 P 7bce241f6ed84e7a91016ce6826fa191 [term 1 FOLLOWER]: Refusing update from remote peer 32e25f10587443fa8727e2306d8ee182: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:43.569157 15003 consensus_queue.cc:1035] T b64d63fe7f20455f8de68c3aade06d11 P 32e25f10587443fa8727e2306d8ee182 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bce241f6ed84e7a91016ce6826fa191" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:07:43.569360 14999 consensus_queue.cc:1035] T b64d63fe7f20455f8de68c3aade06d11 P 32e25f10587443fa8727e2306d8ee182 [LEADER]: Connected to new peer: Peer: permanent_uuid: "489cbb1717784038821fd9f35b855b28" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 40877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:07:43.577811 15007 mvcc.cc:205] Tried to move back new op lower bound from 7317756573967466496 to 7317756573886816256. Current Snapshot: MvccSnapshot[applied={T|T < 7317756573967466496}]
I20260812 19:07:43.587590 14413 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:07:43.587766 14413 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:07:43.590365 14413 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 577626c886304730a53032ddab4923c7: Sending DeleteTablet for 3 replicas of tablet b64d63fe7f20455f8de68c3aade06d11
I20260812 19:07:43.591244 14885 tablet_service.cc:1514] Processing DeleteTablet for tablet b64d63fe7f20455f8de68c3aade06d11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:07:43 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:58230
I20260812 19:07:43.591252 14726 tablet_service.cc:1514] Processing DeleteTablet for tablet b64d63fe7f20455f8de68c3aade06d11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:07:43 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:52458
I20260812 19:07:43.591413 14565 tablet_service.cc:1514] Processing DeleteTablet for tablet b64d63fe7f20455f8de68c3aade06d11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:07:43 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:57184
I20260812 19:07:43.591795 15019 tablet_replica.cc:330] T b64d63fe7f20455f8de68c3aade06d11 P 7bce241f6ed84e7a91016ce6826fa191: stopping tablet replica
I20260812 19:07:43.592052 15020 tablet_replica.cc:330] T b64d63fe7f20455f8de68c3aade06d11 P 32e25f10587443fa8727e2306d8ee182: stopping tablet replica
I20260812 19:07:43.592157 15019 raft_consensus.cc:2229] T b64d63fe7f20455f8de68c3aade06d11 P 7bce241f6ed84e7a91016ce6826fa191 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:43.592344 15017 tablet_replica.cc:330] T b64d63fe7f20455f8de68c3aade06d11 P 489cbb1717784038821fd9f35b855b28: stopping tablet replica
I20260812 19:07:43.592501 15019 raft_consensus.cc:2258] T b64d63fe7f20455f8de68c3aade06d11 P 7bce241f6ed84e7a91016ce6826fa191 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:43.592738 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 14487
I20260812 19:07:43.594623 15019 ts_tablet_manager.cc:1905] T b64d63fe7f20455f8de68c3aade06d11 P 7bce241f6ed84e7a91016ce6826fa191: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:07:43.596679 15017 raft_consensus.cc:2229] T b64d63fe7f20455f8de68c3aade06d11 P 489cbb1717784038821fd9f35b855b28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:43.597170 15019 ts_tablet_manager.cc:1918] T b64d63fe7f20455f8de68c3aade06d11 P 7bce241f6ed84e7a91016ce6826fa191: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:07:43.597272 15019 log.cc:1196] T b64d63fe7f20455f8de68c3aade06d11 P 7bce241f6ed84e7a91016ce6826fa191: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-2/wal/wals/b64d63fe7f20455f8de68c3aade06d11
I20260812 19:07:43.597360 15017 raft_consensus.cc:2258] T b64d63fe7f20455f8de68c3aade06d11 P 489cbb1717784038821fd9f35b855b28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:43.597637 15019 ts_tablet_manager.cc:1939] T b64d63fe7f20455f8de68c3aade06d11 P 7bce241f6ed84e7a91016ce6826fa191: Deleting consensus metadata
I20260812 19:07:43.598965 14395 catalog_manager.cc:4841] TS 7bce241f6ed84e7a91016ce6826fa191 (127.25.130.3:40785): tablet b64d63fe7f20455f8de68c3aade06d11 (table test-table [id=156ed44d41864f4b9fdf82cf9a13118e]) successfully deleted
I20260812 19:07:43.599119 15017 ts_tablet_manager.cc:1905] T b64d63fe7f20455f8de68c3aade06d11 P 489cbb1717784038821fd9f35b855b28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:07:43.601262 15017 ts_tablet_manager.cc:1918] T b64d63fe7f20455f8de68c3aade06d11 P 489cbb1717784038821fd9f35b855b28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:07:43.601343 15017 log.cc:1196] T b64d63fe7f20455f8de68c3aade06d11 P 489cbb1717784038821fd9f35b855b28: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561415988404-26120-0/minicluster-data/ts-1/wal/wals/b64d63fe7f20455f8de68c3aade06d11
I20260812 19:07:43.601624 15017 ts_tablet_manager.cc:1939] T b64d63fe7f20455f8de68c3aade06d11 P 489cbb1717784038821fd9f35b855b28: Deleting consensus metadata
W20260812 19:07:43.602662 14393 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.130.1:38083 (error 108)
I20260812 19:07:43.602730 14394 catalog_manager.cc:4841] TS 489cbb1717784038821fd9f35b855b28 (127.25.130.2:40877): tablet b64d63fe7f20455f8de68c3aade06d11 (table test-table [id=156ed44d41864f4b9fdf82cf9a13118e]) successfully deleted
I20260812 19:07:43.602840 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 14649
I20260812 19:07:43.609877 14983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:43.609509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:38083 (local address 127.0.0.1:57216)
0812 19:07:43.609678 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:43.609772 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.1:38083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260812 19:07:43.610170 14393 catalog_manager.cc:4584] TS 32e25f10587443fa8727e2306d8ee182 (127.25.130.1:38083): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b64d63fe7f20455f8de68c3aade06d11: Network error: Client connection negotiation failed: client connection to 127.25.130.1:38083: connect: Connection refused (error 111)
I20260812 19:07:43.618628 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 14809
I20260812 19:07:43.627395 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 14375
2026-08-12T19:07:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2365 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T19:07:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:07:43Z Disabled control of system clock
I20260812 19:07:43.661198 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:33423
--webserver_interface=127.25.130.62
--webserver_port=0
--builtin_ntp_servers=127.25.130.20:37897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:33423
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 19:07:43.773147 15025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:43.773468 15025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:43.773521 15025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:43.777556 15025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:07:43.777621 15025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:43.777634 15025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:43.777650 15025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:07:43.777663 15025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:07:43.782553 15025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:37897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:33423
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:33423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:43.783641 15025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:43.784983 15025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:43.793340 15032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:43.793399 15035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:43.793340 15033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:43.793632 15025 server_base.cc:1034] running on GCE node
I20260812 19:07:43.794307 15025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:43.795444 15025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:43.796628 15025 hybrid_clock.cc:648] HybridClock initialized: now 1786561663796606 us; error 41 us; skew 500 ppm
I20260812 19:07:43.798651 15025 webserver.cc:460] Webserver started at http://127.25.130.62:33761/ using document root <none> and password file <none>
I20260812 19:07:43.799232 15025 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:43.799294 15025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:43.799515 15025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:43.801612 15025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "fdec3262c0bb434a8b54db7b65514e79"
format_stamp: "Formatted at 2026-08-12 19:07:43 on dist-test-slave-49h3"
server_key: "b1eb55884a1c600d463cfb645d7115e7"
server_key_iv: "fb6cf3d3472c63641012e04c464242b7"
server_key_version: "encryptionkey@0"
I20260812 19:07:43.802175 15025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "fdec3262c0bb434a8b54db7b65514e79"
format_stamp: "Formatted at 2026-08-12 19:07:43 on dist-test-slave-49h3"
server_key: "b1eb55884a1c600d463cfb645d7115e7"
server_key_iv: "fb6cf3d3472c63641012e04c464242b7"
server_key_version: "encryptionkey@0"
I20260812 19:07:43.805846 15025 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 19:07:43.808187 15044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:43.809377 15025 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:07:43.809517 15025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "fdec3262c0bb434a8b54db7b65514e79"
format_stamp: "Formatted at 2026-08-12 19:07:43 on dist-test-slave-49h3"
server_key: "b1eb55884a1c600d463cfb645d7115e7"
server_key_iv: "fb6cf3d3472c63641012e04c464242b7"
server_key_version: "encryptionkey@0"
I20260812 19:07:43.809621 15025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:43.828996 15025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:43.829821 15025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:43.830008 15025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:43.836935 15025 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:33423
I20260812 19:07:43.836956 15105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:33423 every 8 connection(s)
I20260812 19:07:43.837954 15025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:07:43.841180 15107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:43.847044 15107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdec3262c0bb434a8b54db7b65514e79: Bootstrap starting.
I20260812 19:07:43.847520 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 15025
I20260812 19:07:43.847632 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:07:43.847869 26120 external_mini_cluster.cc:1428] Setting key 9bc17fa260364a276c16d14e775b3fcd
I20260812 19:07:43.849807 15107 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdec3262c0bb434a8b54db7b65514e79: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:43.850744 15107 log.cc:824] T 00000000000000000000000000000000 P fdec3262c0bb434a8b54db7b65514e79: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:43.853107 15107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdec3262c0bb434a8b54db7b65514e79: No bootstrap required, opened a new log
I20260812 19:07:43.857481 15107 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdec3262c0bb434a8b54db7b65514e79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdec3262c0bb434a8b54db7b65514e79" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 33423 } }
I20260812 19:07:43.857790 15107 raft_consensus.cc:374] T 00000000000000000000000000000000 P fdec3262c0bb434a8b54db7b65514e79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:43.857851 15107 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdec3262c0bb434a8b54db7b65514e79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdec3262c0bb434a8b54db7b65514e79, State: Initialized, Role: FOLLOWER
I20260812 19:07:43.858354 15107 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdec3262c0bb434a8b54db7b65514e79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdec3262c0bb434a8b54db7b65514e79" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 33423 } }
I20260812 19:07:43.858462 15110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:43.849260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49706 (local address 127.25.130.62:33423)
0812 19:07:43.850016 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:43.850038 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:43.850087 (+ 49us) server_negotiation.cc:407] Connection header received
0812 19:07:43.850961 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:43.850986 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:43.851447 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:43.851813 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:43.852254 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.853720 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.854613 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:43.855334 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:43.855497 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:43.856241 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:43.856248 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:43.856268 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:07:43.856843 (+ 575us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:43.856923 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:07:43.856950 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:43.857053 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:43.857127 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 19:07:43.857415 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":145,"threads_started":1}
I20260812 19:07:43.858520 15107 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdec3262c0bb434a8b54db7b65514e79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:07:43.858579 15107 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdec3262c0bb434a8b54db7b65514e79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:07:43.858677 15107 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdec3262c0bb434a8b54db7b65514e79 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:43.859645 15107 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdec3262c0bb434a8b54db7b65514e79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdec3262c0bb434a8b54db7b65514e79" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 33423 } }
I20260812 19:07:43.859974 15107 leader_election.cc:302] T 00000000000000000000000000000000 P fdec3262c0bb434a8b54db7b65514e79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdec3262c0bb434a8b54db7b65514e79; no voters:
I20260812 19:07:43.860286 15107 leader_election.cc:288] T 00000000000000000000000000000000 P fdec3262c0bb434a8b54db7b65514e79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:07:43.860440 15112 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdec3262c0bb434a8b54db7b65514e79 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:07:43.860730 15112 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdec3262c0bb434a8b54db7b65514e79 [term 1 LEADER]: Becoming Leader. State: Replica: fdec3262c0bb434a8b54db7b65514e79, State: Running, Role: LEADER
I20260812 19:07:43.861066 15112 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdec3262c0bb434a8b54db7b65514e79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdec3262c0bb434a8b54db7b65514e79" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 33423 } }
I20260812 19:07:43.861518 15107 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdec3262c0bb434a8b54db7b65514e79 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:07:43.862656 15114 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdec3262c0bb434a8b54db7b65514e79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdec3262c0bb434a8b54db7b65514e79. Latest consensus state: current_term: 1 leader_uuid: "fdec3262c0bb434a8b54db7b65514e79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdec3262c0bb434a8b54db7b65514e79" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 33423 } } }
I20260812 19:07:43.862802 15114 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdec3262c0bb434a8b54db7b65514e79 [sys.catalog]: This master's current role is: LEADER
I20260812 19:07:43.862668 15113 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdec3262c0bb434a8b54db7b65514e79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdec3262c0bb434a8b54db7b65514e79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdec3262c0bb434a8b54db7b65514e79" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 33423 } } }
I20260812 19:07:43.862962 15113 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdec3262c0bb434a8b54db7b65514e79 [sys.catalog]: This master's current role is: LEADER
I20260812 19:07:43.863256 15122 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:07:43.866622 15122 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:07:43.872529 15122 catalog_manager.cc:1347] Generated new cluster ID: befd1b18ad65446eab5373af12b1b5af
I20260812 19:07:43.872620 15122 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:07:43.880175 15122 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:07:43.881327 15122 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:07:43.889191 15122 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fdec3262c0bb434a8b54db7b65514e79: Generated new TSK 0
I20260812 19:07:43.889935 15122 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:07:43.893038 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:0
--local_ip_for_outbound_sockets=127.25.130.1
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:33423
--builtin_ntp_servers=127.25.130.20:37897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 19:07:44.009670 15133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:44.009934 15133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:44.009974 15133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:44.014070 15133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:44.014129 15133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:44.014184 15133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:07:44.018961 15133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:37897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:33423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:44.020042 15133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:44.021418 15133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:44.031204 15140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:44.031227 15141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:44.031216 15143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:44.031440 15133 server_base.cc:1034] running on GCE node
I20260812 19:07:44.031976 15133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:44.032677 15133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:44.033883 15133 hybrid_clock.cc:648] HybridClock initialized: now 1786561664033861 us; error 38 us; skew 500 ppm
I20260812 19:07:44.035884 15133 webserver.cc:460] Webserver started at http://127.25.130.1:41347/ using document root <none> and password file <none>
I20260812 19:07:44.036460 15133 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:44.036545 15133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:44.036765 15133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:44.039163 15133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-0/data/instance:
uuid: "30d265e202e0489f9eb9266050e4b4fb"
format_stamp: "Formatted at 2026-08-12 19:07:44 on dist-test-slave-49h3"
server_key: "d3775c9a29b4027aa582ba6f616e3ae7"
server_key_iv: "1640a030125e870ec8d31d3cc7f0573e"
server_key_version: "encryptionkey@0"
I20260812 19:07:44.039790 15133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance:
uuid: "30d265e202e0489f9eb9266050e4b4fb"
format_stamp: "Formatted at 2026-08-12 19:07:44 on dist-test-slave-49h3"
server_key: "d3775c9a29b4027aa582ba6f616e3ae7"
server_key_iv: "1640a030125e870ec8d31d3cc7f0573e"
server_key_version: "encryptionkey@0"
I20260812 19:07:44.043742 15133 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 19:07:44.046423 15150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:44.047704 15133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 19:07:44.047825 15133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "30d265e202e0489f9eb9266050e4b4fb"
format_stamp: "Formatted at 2026-08-12 19:07:44 on dist-test-slave-49h3"
server_key: "d3775c9a29b4027aa582ba6f616e3ae7"
server_key_iv: "1640a030125e870ec8d31d3cc7f0573e"
server_key_version: "encryptionkey@0"
I20260812 19:07:44.047933 15133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:44.071367 15133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:44.072124 15133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:44.072309 15133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:44.072934 15133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:07:44.074138 15133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:07:44.074200 15133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:44.074268 15133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:07:44.074294 15133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:44.083401 15133 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:41731
I20260812 19:07:44.083423 15290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:41731 every 8 connection(s)
I20260812 19:07:44.084343 15133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:07:44.090642 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 15133
I20260812 19:07:44.090741 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance
I20260812 19:07:44.091002 26120 external_mini_cluster.cc:1428] Setting key f95d76b0039e28508fa890454b4410cd
I20260812 19:07:44.092905 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:0
--local_ip_for_outbound_sockets=127.25.130.2
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:33423
--builtin_ntp_servers=127.25.130.20:37897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 19:07:44.093668 15110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:44.086187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:34215 (local address 127.25.130.62:33423)
0812 19:07:44.086335 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:44.086339 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:44.087446 (+ 1107us) server_negotiation.cc:407] Connection header received
0812 19:07:44.088824 (+ 1378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:44.088828 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:44.088886 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:44.088961 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:44.089915 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.090454 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.091678 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.091906 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.091998 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:44.093145 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:44.093149 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:44.093151 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:07:44.093163 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:44.093184 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:07:44.093192 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:44.093227 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:44.093422 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 19:07:44.093507 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 19:07:44.094537 15293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:44.086604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33423 (local address 127.25.130.1:34215)
0812 19:07:44.087297 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:44.087338 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:44.088603 (+ 1265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:44.089150 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:44.089157 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:44.089241 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:07:44.089770 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:44.089784 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.090586 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.090588 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:44.091545 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:44.091557 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.092070 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.092075 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:44.092232 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:44.092836 (+ 604us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:07:44.092837 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:07:44.092879 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:07:44.092949 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:07:44.093279 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:07:44.093287 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:07:44.093291 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 19:07:44.093392 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 19:07:44.093717 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":137,"threads_started":1}
I20260812 19:07:44.095777 15291 heartbeater.cc:347] Connected to a master server at 127.25.130.62:33423
I20260812 19:07:44.096022 15291 heartbeater.cc:464] Registering TS with master...
I20260812 19:07:44.096577 15291 heartbeater.cc:511] Master 127.25.130.62:33423 requested a full tablet report, sending...
I20260812 19:07:44.098243 15067 ts_manager.cc:194] Registered new tserver with Master: 30d265e202e0489f9eb9266050e4b4fb (127.25.130.1:41731)
I20260812 19:07:44.099433 15067 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.130.1:34215
W20260812 19:07:44.209548 15294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:44.209846 15294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:44.209888 15294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:44.214155 15294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:44.214231 15294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:44.214295 15294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:07:44.219264 15294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:37897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:33423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:44.220464 15294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:44.222003 15294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:44.231786 15302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:44.231806 15304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:44.231787 15301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:44.232131 15294 server_base.cc:1034] running on GCE node
I20260812 19:07:44.232537 15294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:44.233199 15294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:44.234355 15294 hybrid_clock.cc:648] HybridClock initialized: now 1786561664234323 us; error 49 us; skew 500 ppm
I20260812 19:07:44.236184 15294 webserver.cc:460] Webserver started at http://127.25.130.2:34439/ using document root <none> and password file <none>
I20260812 19:07:44.236765 15294 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:44.236836 15294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:44.237048 15294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:44.239207 15294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-1/data/instance:
uuid: "56959f2c2e114e58b05f31f25ef3633c"
format_stamp: "Formatted at 2026-08-12 19:07:44 on dist-test-slave-49h3"
server_key: "c76870586f0e5f934ea771557b4522af"
server_key_iv: "acc62d105062c021495a531b0f0300f7"
server_key_version: "encryptionkey@0"
I20260812 19:07:44.239791 15294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance:
uuid: "56959f2c2e114e58b05f31f25ef3633c"
format_stamp: "Formatted at 2026-08-12 19:07:44 on dist-test-slave-49h3"
server_key: "c76870586f0e5f934ea771557b4522af"
server_key_iv: "acc62d105062c021495a531b0f0300f7"
server_key_version: "encryptionkey@0"
I20260812 19:07:44.243598 15294 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260812 19:07:44.246094 15312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:44.247273 15294 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:07:44.247388 15294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "56959f2c2e114e58b05f31f25ef3633c"
format_stamp: "Formatted at 2026-08-12 19:07:44 on dist-test-slave-49h3"
server_key: "c76870586f0e5f934ea771557b4522af"
server_key_iv: "acc62d105062c021495a531b0f0300f7"
server_key_version: "encryptionkey@0"
I20260812 19:07:44.247491 15294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:44.260164 15294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:44.260885 15294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:44.261063 15294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:44.261734 15294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:07:44.262737 15294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:07:44.262789 15294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:44.262842 15294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:07:44.262871 15294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:44.271356 15294 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:39775
I20260812 19:07:44.271386 15458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:39775 every 8 connection(s)
I20260812 19:07:44.272285 15294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:07:44.281195 15110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:44.274158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:48285 (local address 127.25.130.62:33423)
0812 19:07:44.274322 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:44.274326 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:44.275386 (+ 1060us) server_negotiation.cc:407] Connection header received
0812 19:07:44.276758 (+ 1372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:44.276761 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:44.276814 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:44.276891 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:44.277801 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.278373 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.279452 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.279676 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.279756 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:44.280790 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:44.280793 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:44.280796 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:07:44.280810 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:44.280831 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:07:44.280839 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:44.280891 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:44.280999 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 19:07:44.281073 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 19:07:44.281910 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 15294
I20260812 19:07:44.282013 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance
I20260812 19:07:44.282160 15461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:44.274585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33423 (local address 127.25.130.2:48285)
0812 19:07:44.275238 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:44.275280 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:44.276537 (+ 1257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:44.277047 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:44.277053 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:44.277139 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:07:44.277664 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:44.277679 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.278468 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.278470 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:44.279338 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:44.279348 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.279800 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.279803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:44.279943 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:44.280528 (+ 585us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:07:44.280530 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:07:44.280570 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:07:44.280636 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:07:44.280899 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:07:44.280904 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:07:44.280907 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 19:07:44.280995 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 19:07:44.281223 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":120,"threads_started":1}
I20260812 19:07:44.282305 26120 external_mini_cluster.cc:1428] Setting key ed425a72452475b9648d5b7f516f0885
I20260812 19:07:44.283388 15459 heartbeater.cc:347] Connected to a master server at 127.25.130.62:33423
I20260812 19:07:44.283643 15459 heartbeater.cc:464] Registering TS with master...
I20260812 19:07:44.284119 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:0
--local_ip_for_outbound_sockets=127.25.130.3
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:33423
--builtin_ntp_servers=127.25.130.20:37897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 19:07:44.284162 15459 heartbeater.cc:511] Master 127.25.130.62:33423 requested a full tablet report, sending...
I20260812 19:07:44.285192 15067 ts_manager.cc:194] Registered new tserver with Master: 56959f2c2e114e58b05f31f25ef3633c (127.25.130.2:39775)
I20260812 19:07:44.285806 15067 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.130.2:48285
W20260812 19:07:44.398496 15462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:44.398777 15462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:44.398815 15462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:44.402961 15462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:44.403050 15462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:44.403107 15462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:07:44.407976 15462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:37897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:33423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:44.409108 15462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:44.410625 15462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:44.420606 15471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:44.420698 15462 server_base.cc:1034] running on GCE node
W20260812 19:07:44.420609 15467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:44.420606 15468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:44.421366 15462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:44.422075 15462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:44.423233 15462 hybrid_clock.cc:648] HybridClock initialized: now 1786561664423213 us; error 39 us; skew 500 ppm
I20260812 19:07:44.425117 15462 webserver.cc:460] Webserver started at http://127.25.130.3:44257/ using document root <none> and password file <none>
I20260812 19:07:44.425732 15462 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:44.425806 15462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:44.426033 15462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:44.428129 15462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-2/data/instance:
uuid: "8d70ddb7502d4be29c144f67b095d76a"
format_stamp: "Formatted at 2026-08-12 19:07:44 on dist-test-slave-49h3"
server_key: "aad3ab11eb8e3dba7cd5b572c52c8025"
server_key_iv: "412efbc2f768bc07c13fe6c40a6e69e3"
server_key_version: "encryptionkey@0"
I20260812 19:07:44.428710 15462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance:
uuid: "8d70ddb7502d4be29c144f67b095d76a"
format_stamp: "Formatted at 2026-08-12 19:07:44 on dist-test-slave-49h3"
server_key: "aad3ab11eb8e3dba7cd5b572c52c8025"
server_key_iv: "412efbc2f768bc07c13fe6c40a6e69e3"
server_key_version: "encryptionkey@0"
I20260812 19:07:44.432503 15462 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 19:07:44.435041 15477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:44.436313 15462 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 19:07:44.436434 15462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "8d70ddb7502d4be29c144f67b095d76a"
format_stamp: "Formatted at 2026-08-12 19:07:44 on dist-test-slave-49h3"
server_key: "aad3ab11eb8e3dba7cd5b572c52c8025"
server_key_iv: "412efbc2f768bc07c13fe6c40a6e69e3"
server_key_version: "encryptionkey@0"
I20260812 19:07:44.436543 15462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:44.451294 15462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:44.452055 15462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:44.452240 15462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:44.452852 15462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:07:44.453852 15462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:07:44.453903 15462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:44.453958 15462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:07:44.453985 15462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:44.462309 15462 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:40739
I20260812 19:07:44.462329 15614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:40739 every 8 connection(s)
I20260812 19:07:44.463327 15462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:07:44.472180 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 15462
I20260812 19:07:44.472275 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance
I20260812 19:07:44.472525 26120 external_mini_cluster.cc:1428] Setting key 80f9813bc1a4179056ff9f58ef06aa0f
I20260812 19:07:44.472585 15110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:44.465213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:59187 (local address 127.25.130.62:33423)
0812 19:07:44.465356 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:44.465360 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:44.466533 (+ 1173us) server_negotiation.cc:407] Connection header received
0812 19:07:44.467902 (+ 1369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:44.467906 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:44.467968 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:44.468041 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:44.468949 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.469525 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.470682 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.470911 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.471002 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:44.472095 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:44.472098 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:44.472100 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:07:44.472113 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:44.472133 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:07:44.472141 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:44.472199 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:44.472356 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 19:07:44.472423 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 19:07:44.473623 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:44.465712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33423 (local address 127.25.130.3:59187)
0812 19:07:44.466380 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:44.466422 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:44.467693 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:44.468232 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:44.468238 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:44.468325 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:07:44.468808 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:44.468821 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.469658 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.469661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:44.470554 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:44.470564 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.471068 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.471072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:44.471238 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:44.471829 (+ 591us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:07:44.471831 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:07:44.471871 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:07:44.471938 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:07:44.472211 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:07:44.472221 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:07:44.472227 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 19:07:44.472348 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 19:07:44.472634 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":103,"threads_started":1}
I20260812 19:07:44.474880 15615 heartbeater.cc:347] Connected to a master server at 127.25.130.62:33423
I20260812 19:07:44.475091 15615 heartbeater.cc:464] Registering TS with master...
I20260812 19:07:44.475565 15615 heartbeater.cc:511] Master 127.25.130.62:33423 requested a full tablet report, sending...
I20260812 19:07:44.476511 15067 ts_manager.cc:194] Registered new tserver with Master: 8d70ddb7502d4be29c144f67b095d76a (127.25.130.3:40739)
I20260812 19:07:44.477082 15067 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.130.3:59187
I20260812 19:07:44.486500 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:07:44.491578 15110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:44.488377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49718 (local address 127.25.130.62:33423)
0812 19:07:44.488494 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:44.488497 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:44.488641 (+ 144us) server_negotiation.cc:407] Connection header received
0812 19:07:44.488750 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:44.488752 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:44.488790 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:44.488855 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:44.489057 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.489606 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.490358 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.490560 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.490628 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:44.491265 (+ 637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:44.491268 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:44.491270 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:07:44.491282 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:44.491301 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:07:44.491309 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:44.491360 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:44.491415 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 19:07:44.491466 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 19:07:44.494243 15067 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:07:44.497252 15067 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:07:44.511848 15629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:44.505863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:39775 (local address 127.0.0.1:37276)
0812 19:07:44.506612 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:44.506624 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:44.506750 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:44.507518 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:44.507524 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:44.507548 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:44.507680 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:44.507686 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.508879 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.508882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:44.510206 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:44.510218 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.511265 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.511269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:44.511395 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:44.511435 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 19:07:44.511543 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 19:07:44.511680 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"spinlock_wait_cycles":2816,"thread_start_us":112,"threads_started":1}
I20260812 19:07:44.511875 15630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:44.505710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:41731 (local address 127.0.0.1:51198)
0812 19:07:44.506548 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:44.506599 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:44.506750 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:44.507836 (+ 1086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:44.507838 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:44.507857 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:44.507956 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:44.507962 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.509355 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.509358 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:44.510546 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:44.510556 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.511562 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.511565 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:44.511648 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:44.511669 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 19:07:44.511723 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 19:07:44.511777 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"mutex_wait_us":1,"spinlock_wait_cycles":5888,"thread_start_us":95,"threads_started":1}
I20260812 19:07:44.512327 15633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:44.506029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37276 (local address 127.25.130.2:39775)
0812 19:07:44.506484 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:44.506496 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:44.506726 (+ 230us) server_negotiation.cc:407] Connection header received
0812 19:07:44.506940 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:44.506947 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:44.507174 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:44.507304 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:44.507803 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.508779 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.510350 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.511143 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.511248 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:44.511461 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:44.512060 (+ 599us) server_negotiation.cc:299] Negotiation successful
0812 19:07:44.512165 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":150,"threads_started":1}
I20260812 19:07:44.512548 15631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:44.505836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51198 (local address 127.25.130.1:41731)
0812 19:07:44.507422 (+ 1586us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:44.507435 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:44.507458 (+ 23us) server_negotiation.cc:407] Connection header received
0812 19:07:44.507529 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:44.507534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:44.507715 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:44.507853 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:44.508065 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.509226 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.510707 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.511460 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.511564 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:44.511759 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:44.511816 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 19:07:44.512283 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1115,"thread_start_us":173,"threads_started":1}
I20260812 19:07:44.513051 15637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:44.507073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58302 (local address 127.25.130.3:40739)
0812 19:07:44.507974 (+ 901us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:44.507985 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:44.508008 (+ 23us) server_negotiation.cc:407] Connection header received
0812 19:07:44.508084 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:44.508088 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:44.508273 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:44.508416 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:44.508619 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.509730 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.511053 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.511814 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.511908 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:44.512079 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:44.512801 (+ 722us) server_negotiation.cc:299] Negotiation successful
0812 19:07:44.512895 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":800,"thread_start_us":101,"threads_started":1}
I20260812 19:07:44.513428 15634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:44.506561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:40739 (local address 127.0.0.1:58302)
0812 19:07:44.507241 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:44.507255 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:44.507339 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:44.508409 (+ 1070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:44.508412 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:44.508430 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:44.508517 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:44.508523 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.509858 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.509861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:44.510925 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:44.510934 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.512140 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.512144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:44.512239 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:44.512263 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 19:07:44.512325 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 19:07:44.512395 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":89,"threads_started":1}
I20260812 19:07:44.515352 15210 tablet_service.cc:1467] Processing CreateTablet for tablet 359ab7b7afa0403a9ee4d6af509c6c9e (DEFAULT_TABLE table=test-table [id=fc70f8e8b05e408e976dd6c574ecf4fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:07:44.515448 15371 tablet_service.cc:1467] Processing CreateTablet for tablet 359ab7b7afa0403a9ee4d6af509c6c9e (DEFAULT_TABLE table=test-table [id=fc70f8e8b05e408e976dd6c574ecf4fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:07:44.515874 15540 tablet_service.cc:1467] Processing CreateTablet for tablet 359ab7b7afa0403a9ee4d6af509c6c9e (DEFAULT_TABLE table=test-table [id=fc70f8e8b05e408e976dd6c574ecf4fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:07:44.516271 15210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 359ab7b7afa0403a9ee4d6af509c6c9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:44.516476 15371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 359ab7b7afa0403a9ee4d6af509c6c9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:44.516737 15540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 359ab7b7afa0403a9ee4d6af509c6c9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:44.523216 15638 tablet_bootstrap.cc:492] T 359ab7b7afa0403a9ee4d6af509c6c9e P 30d265e202e0489f9eb9266050e4b4fb: Bootstrap starting.
I20260812 19:07:44.523820 15640 tablet_bootstrap.cc:492] T 359ab7b7afa0403a9ee4d6af509c6c9e P 8d70ddb7502d4be29c144f67b095d76a: Bootstrap starting.
I20260812 19:07:44.523820 15639 tablet_bootstrap.cc:492] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c: Bootstrap starting.
I20260812 19:07:44.525419 15638 tablet_bootstrap.cc:654] T 359ab7b7afa0403a9ee4d6af509c6c9e P 30d265e202e0489f9eb9266050e4b4fb: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:44.525949 15640 tablet_bootstrap.cc:654] T 359ab7b7afa0403a9ee4d6af509c6c9e P 8d70ddb7502d4be29c144f67b095d76a: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:44.525949 15639 tablet_bootstrap.cc:654] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:44.526237 15638 log.cc:824] T 359ab7b7afa0403a9ee4d6af509c6c9e P 30d265e202e0489f9eb9266050e4b4fb: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:44.526784 15639 log.cc:824] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:44.526784 15640 log.cc:824] T 359ab7b7afa0403a9ee4d6af509c6c9e P 8d70ddb7502d4be29c144f67b095d76a: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:44.528309 15638 tablet_bootstrap.cc:492] T 359ab7b7afa0403a9ee4d6af509c6c9e P 30d265e202e0489f9eb9266050e4b4fb: No bootstrap required, opened a new log
I20260812 19:07:44.528519 15638 ts_tablet_manager.cc:1397] T 359ab7b7afa0403a9ee4d6af509c6c9e P 30d265e202e0489f9eb9266050e4b4fb: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 19:07:44.529297 15639 tablet_bootstrap.cc:492] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c: No bootstrap required, opened a new log
I20260812 19:07:44.529501 15640 tablet_bootstrap.cc:492] T 359ab7b7afa0403a9ee4d6af509c6c9e P 8d70ddb7502d4be29c144f67b095d76a: No bootstrap required, opened a new log
I20260812 19:07:44.529520 15639 ts_tablet_manager.cc:1397] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 19:07:44.529695 15640 ts_tablet_manager.cc:1397] T 359ab7b7afa0403a9ee4d6af509c6c9e P 8d70ddb7502d4be29c144f67b095d76a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 19:07:44.532735 15640 raft_consensus.cc:348] T 359ab7b7afa0403a9ee4d6af509c6c9e P 8d70ddb7502d4be29c144f67b095d76a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d265e202e0489f9eb9266050e4b4fb" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 41731 } } peers { permanent_uuid: "56959f2c2e114e58b05f31f25ef3633c" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 39775 } } peers { permanent_uuid: "8d70ddb7502d4be29c144f67b095d76a" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40739 } }
I20260812 19:07:44.532984 15640 raft_consensus.cc:374] T 359ab7b7afa0403a9ee4d6af509c6c9e P 8d70ddb7502d4be29c144f67b095d76a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:44.533037 15640 raft_consensus.cc:729] T 359ab7b7afa0403a9ee4d6af509c6c9e P 8d70ddb7502d4be29c144f67b095d76a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d70ddb7502d4be29c144f67b095d76a, State: Initialized, Role: FOLLOWER
I20260812 19:07:44.532846 15638 raft_consensus.cc:348] T 359ab7b7afa0403a9ee4d6af509c6c9e P 30d265e202e0489f9eb9266050e4b4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d265e202e0489f9eb9266050e4b4fb" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 41731 } } peers { permanent_uuid: "56959f2c2e114e58b05f31f25ef3633c" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 39775 } } peers { permanent_uuid: "8d70ddb7502d4be29c144f67b095d76a" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40739 } }
I20260812 19:07:44.533110 15638 raft_consensus.cc:374] T 359ab7b7afa0403a9ee4d6af509c6c9e P 30d265e202e0489f9eb9266050e4b4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:44.533172 15638 raft_consensus.cc:729] T 359ab7b7afa0403a9ee4d6af509c6c9e P 30d265e202e0489f9eb9266050e4b4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d265e202e0489f9eb9266050e4b4fb, State: Initialized, Role: FOLLOWER
I20260812 19:07:44.533563 15640 consensus_queue.cc:260] T 359ab7b7afa0403a9ee4d6af509c6c9e P 8d70ddb7502d4be29c144f67b095d76a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d265e202e0489f9eb9266050e4b4fb" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 41731 } } peers { permanent_uuid: "56959f2c2e114e58b05f31f25ef3633c" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 39775 } } peers { permanent_uuid: "8d70ddb7502d4be29c144f67b095d76a" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40739 } }
I20260812 19:07:44.533617 15638 consensus_queue.cc:260] T 359ab7b7afa0403a9ee4d6af509c6c9e P 30d265e202e0489f9eb9266050e4b4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d265e202e0489f9eb9266050e4b4fb" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 41731 } } peers { permanent_uuid: "56959f2c2e114e58b05f31f25ef3633c" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 39775 } } peers { permanent_uuid: "8d70ddb7502d4be29c144f67b095d76a" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40739 } }
I20260812 19:07:44.533556 15639 raft_consensus.cc:348] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d265e202e0489f9eb9266050e4b4fb" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 41731 } } peers { permanent_uuid: "56959f2c2e114e58b05f31f25ef3633c" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 39775 } } peers { permanent_uuid: "8d70ddb7502d4be29c144f67b095d76a" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40739 } }
I20260812 19:07:44.533802 15639 raft_consensus.cc:374] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:44.533861 15639 raft_consensus.cc:729] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56959f2c2e114e58b05f31f25ef3633c, State: Initialized, Role: FOLLOWER
I20260812 19:07:44.534284 15615 heartbeater.cc:503] Master 127.25.130.62:33423 was elected leader, sending a full tablet report...
I20260812 19:07:44.534296 15639 consensus_queue.cc:260] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d265e202e0489f9eb9266050e4b4fb" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 41731 } } peers { permanent_uuid: "56959f2c2e114e58b05f31f25ef3633c" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 39775 } } peers { permanent_uuid: "8d70ddb7502d4be29c144f67b095d76a" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40739 } }
I20260812 19:07:44.534606 15640 ts_tablet_manager.cc:1428] T 359ab7b7afa0403a9ee4d6af509c6c9e P 8d70ddb7502d4be29c144f67b095d76a: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260812 19:07:44.534608 15638 ts_tablet_manager.cc:1428] T 359ab7b7afa0403a9ee4d6af509c6c9e P 30d265e202e0489f9eb9266050e4b4fb: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 19:07:44.534946 15291 heartbeater.cc:503] Master 127.25.130.62:33423 was elected leader, sending a full tablet report...
I20260812 19:07:44.535152 15639 ts_tablet_manager.cc:1428] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260812 19:07:44.535450 15459 heartbeater.cc:503] Master 127.25.130.62:33423 was elected leader, sending a full tablet report...
W20260812 19:07:44.585558 15292 tablet.cc:2369] T 359ab7b7afa0403a9ee4d6af509c6c9e P 30d265e202e0489f9eb9266050e4b4fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:07:44.682086 15647 raft_consensus.cc:482] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:44.682310 15647 raft_consensus.cc:504] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d265e202e0489f9eb9266050e4b4fb" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 41731 } } peers { permanent_uuid: "56959f2c2e114e58b05f31f25ef3633c" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 39775 } } peers { permanent_uuid: "8d70ddb7502d4be29c144f67b095d76a" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40739 } }
I20260812 19:07:44.683513 15647 leader_election.cc:288] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d265e202e0489f9eb9266050e4b4fb (127.25.130.1:41731), 8d70ddb7502d4be29c144f67b095d76a (127.25.130.3:40739)
I20260812 19:07:44.687049 15461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:44.683781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:40739 (local address 127.25.130.2:50937)
0812 19:07:44.683916 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:44.683932 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:44.684042 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:44.684325 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:44.684328 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:44.684346 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:44.684456 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:44.684462 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.685165 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.685169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:44.685977 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:44.685986 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.686620 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.686624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:44.686747 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:44.686766 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 19:07:44.686831 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 19:07:44.686901 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 19:07:44.687049 15637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:44.683892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:50937 (local address 127.25.130.3:40739)
0812 19:07:44.684024 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:44.684030 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:44.684048 (+ 18us) server_negotiation.cc:407] Connection header received
0812 19:07:44.684174 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:44.684178 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:44.684222 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:44.684297 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:44.684564 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.685056 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.686108 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.686511 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.686620 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:44.686674 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:44.686856 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 19:07:44.686921 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 19:07:44.687278 15648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:44.683781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:41731 (local address 127.25.130.2:51193)
0812 19:07:44.684203 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:44.684214 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:44.684301 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:44.684519 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:44.684522 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:44.684537 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:44.684659 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:44.684664 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.685378 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.685381 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:44.686198 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:44.686207 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.686823 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.686827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:44.686928 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:44.686967 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 19:07:44.687067 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 19:07:44.687133 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":126,"thread_start_us":120,"threads_started":1}
I20260812 19:07:44.687268 15631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:44.683915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:51193 (local address 127.25.130.1:41731)
0812 19:07:44.684035 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:44.684039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:44.684295 (+ 256us) server_negotiation.cc:407] Connection header received
0812 19:07:44.684389 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:44.684392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:44.684438 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:44.684497 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:44.684768 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.685271 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.686321 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.686712 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.686812 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:44.686861 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:44.687087 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 19:07:44.687164 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 19:07:44.687767 15562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "359ab7b7afa0403a9ee4d6af509c6c9e" candidate_uuid: "56959f2c2e114e58b05f31f25ef3633c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d70ddb7502d4be29c144f67b095d76a" is_pre_election: true
I20260812 19:07:44.687945 15233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "359ab7b7afa0403a9ee4d6af509c6c9e" candidate_uuid: "56959f2c2e114e58b05f31f25ef3633c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d265e202e0489f9eb9266050e4b4fb" is_pre_election: true
I20260812 19:07:44.688138 15562 raft_consensus.cc:2454] T 359ab7b7afa0403a9ee4d6af509c6c9e P 8d70ddb7502d4be29c144f67b095d76a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56959f2c2e114e58b05f31f25ef3633c in term 0.
I20260812 19:07:44.688228 15233 raft_consensus.cc:2454] T 359ab7b7afa0403a9ee4d6af509c6c9e P 30d265e202e0489f9eb9266050e4b4fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56959f2c2e114e58b05f31f25ef3633c in term 0.
I20260812 19:07:44.688679 15319 leader_election.cc:302] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 30d265e202e0489f9eb9266050e4b4fb, 56959f2c2e114e58b05f31f25ef3633c, 8d70ddb7502d4be29c144f67b095d76a; no voters:
I20260812 19:07:44.688935 15647 raft_consensus.cc:2781] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:07:44.689015 15647 raft_consensus.cc:482] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:44.689062 15647 raft_consensus.cc:3037] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:44.690119 15647 raft_consensus.cc:504] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d265e202e0489f9eb9266050e4b4fb" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 41731 } } peers { permanent_uuid: "56959f2c2e114e58b05f31f25ef3633c" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 39775 } } peers { permanent_uuid: "8d70ddb7502d4be29c144f67b095d76a" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40739 } }
I20260812 19:07:44.690454 15647 leader_election.cc:288] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c [CANDIDATE]: Term 1 election: Requested vote from peers 30d265e202e0489f9eb9266050e4b4fb (127.25.130.1:41731), 8d70ddb7502d4be29c144f67b095d76a (127.25.130.3:40739)
I20260812 19:07:44.690774 15233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "359ab7b7afa0403a9ee4d6af509c6c9e" candidate_uuid: "56959f2c2e114e58b05f31f25ef3633c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d265e202e0489f9eb9266050e4b4fb"
I20260812 19:07:44.690894 15233 raft_consensus.cc:3037] T 359ab7b7afa0403a9ee4d6af509c6c9e P 30d265e202e0489f9eb9266050e4b4fb [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:44.690958 15562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "359ab7b7afa0403a9ee4d6af509c6c9e" candidate_uuid: "56959f2c2e114e58b05f31f25ef3633c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d70ddb7502d4be29c144f67b095d76a"
I20260812 19:07:44.691083 15562 raft_consensus.cc:3037] T 359ab7b7afa0403a9ee4d6af509c6c9e P 8d70ddb7502d4be29c144f67b095d76a [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:44.691946 15233 raft_consensus.cc:2454] T 359ab7b7afa0403a9ee4d6af509c6c9e P 30d265e202e0489f9eb9266050e4b4fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56959f2c2e114e58b05f31f25ef3633c in term 1.
I20260812 19:07:44.692029 15562 raft_consensus.cc:2454] T 359ab7b7afa0403a9ee4d6af509c6c9e P 8d70ddb7502d4be29c144f67b095d76a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56959f2c2e114e58b05f31f25ef3633c in term 1.
I20260812 19:07:44.692253 15313 leader_election.cc:302] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d265e202e0489f9eb9266050e4b4fb, 56959f2c2e114e58b05f31f25ef3633c; no voters:
I20260812 19:07:44.692436 15647 raft_consensus.cc:2781] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:07:44.692638 15647 raft_consensus.cc:686] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c [term 1 LEADER]: Becoming Leader. State: Replica: 56959f2c2e114e58b05f31f25ef3633c, State: Running, Role: LEADER
I20260812 19:07:44.692981 15647 consensus_queue.cc:237] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d265e202e0489f9eb9266050e4b4fb" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 41731 } } peers { permanent_uuid: "56959f2c2e114e58b05f31f25ef3633c" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 39775 } } peers { permanent_uuid: "8d70ddb7502d4be29c144f67b095d76a" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40739 } }
I20260812 19:07:44.696090 15067 catalog_manager.cc:5495] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c reported cstate change: term changed from 0 to 1, leader changed from <none> to 56959f2c2e114e58b05f31f25ef3633c (127.25.130.2). New cstate: current_term: 1 leader_uuid: "56959f2c2e114e58b05f31f25ef3633c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30d265e202e0489f9eb9266050e4b4fb" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 41731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56959f2c2e114e58b05f31f25ef3633c" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 39775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d70ddb7502d4be29c144f67b095d76a" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40739 } health_report { overall_health: UNKNOWN } } }
W20260812 19:07:44.714329 15616 tablet.cc:2369] T 359ab7b7afa0403a9ee4d6af509c6c9e P 8d70ddb7502d4be29c144f67b095d76a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:07:44.747547 15633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:44.744377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37288 (local address 127.25.130.2:39775)
0812 19:07:44.744507 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:44.744510 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:44.744524 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:07:44.744605 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:44.744607 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:44.744674 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:44.744761 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:07:44.744949 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.745477 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.746309 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:44.746513 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:44.746581 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:44.746806 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:07:44.747229 (+ 423us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:07:44.747318 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:44.747342 (+ 24us) server_negotiation.cc:299] Negotiation successful
0812 19:07:44.747389 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 19:07:44.753400 15233 raft_consensus.cc:1261] T 359ab7b7afa0403a9ee4d6af509c6c9e P 30d265e202e0489f9eb9266050e4b4fb [term 1 FOLLOWER]: Refusing update from remote peer 56959f2c2e114e58b05f31f25ef3633c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:44.753400 15562 raft_consensus.cc:1261] T 359ab7b7afa0403a9ee4d6af509c6c9e P 8d70ddb7502d4be29c144f67b095d76a [term 1 FOLLOWER]: Refusing update from remote peer 56959f2c2e114e58b05f31f25ef3633c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:44.754149 15647 consensus_queue.cc:1035] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d265e202e0489f9eb9266050e4b4fb" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 41731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:07:44.754346 15649 consensus_queue.cc:1035] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d70ddb7502d4be29c144f67b095d76a" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 40739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:07:44.771296 15067 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:07:44.771517 15067 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:07:44.774430 15067 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fdec3262c0bb434a8b54db7b65514e79: Sending DeleteTablet for 3 replicas of tablet 359ab7b7afa0403a9ee4d6af509c6c9e
I20260812 19:07:44.775177 15371 tablet_service.cc:1514] Processing DeleteTablet for tablet 359ab7b7afa0403a9ee4d6af509c6c9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:07:44 UTC) from {username='slave'} at 127.0.0.1:37276
I20260812 19:07:44.775210 15210 tablet_service.cc:1514] Processing DeleteTablet for tablet 359ab7b7afa0403a9ee4d6af509c6c9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:07:44 UTC) from {username='slave'} at 127.0.0.1:51198
I20260812 19:07:44.775669 15665 tablet_replica.cc:330] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c: stopping tablet replica
I20260812 19:07:44.775717 15540 tablet_service.cc:1514] Processing DeleteTablet for tablet 359ab7b7afa0403a9ee4d6af509c6c9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:07:44 UTC) from {username='slave'} at 127.0.0.1:58302
I20260812 19:07:44.775990 15665 raft_consensus.cc:2229] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:07:44.776230 15667 tablet_replica.cc:330] T 359ab7b7afa0403a9ee4d6af509c6c9e P 8d70ddb7502d4be29c144f67b095d76a: stopping tablet replica
I20260812 19:07:44.776423 15665 raft_consensus.cc:2258] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:44.776633 15667 raft_consensus.cc:2229] T 359ab7b7afa0403a9ee4d6af509c6c9e P 8d70ddb7502d4be29c144f67b095d76a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:44.776883 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 15133
I20260812 19:07:44.786629 15667 raft_consensus.cc:2258] T 359ab7b7afa0403a9ee4d6af509c6c9e P 8d70ddb7502d4be29c144f67b095d76a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:44.790037 15667 ts_tablet_manager.cc:1905] T 359ab7b7afa0403a9ee4d6af509c6c9e P 8d70ddb7502d4be29c144f67b095d76a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:07:44.792939 15667 ts_tablet_manager.cc:1918] T 359ab7b7afa0403a9ee4d6af509c6c9e P 8d70ddb7502d4be29c144f67b095d76a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:07:44.793116 15667 log.cc:1196] T 359ab7b7afa0403a9ee4d6af509c6c9e P 8d70ddb7502d4be29c144f67b095d76a: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561415988404-26120-0/minicluster-data/ts-2/wal/wals/359ab7b7afa0403a9ee4d6af509c6c9e
I20260812 19:07:44.793534 15667 ts_tablet_manager.cc:1939] T 359ab7b7afa0403a9ee4d6af509c6c9e P 8d70ddb7502d4be29c144f67b095d76a: Deleting consensus metadata
I20260812 19:07:44.794943 15053 catalog_manager.cc:4841] TS 8d70ddb7502d4be29c144f67b095d76a (127.25.130.3:40739): tablet 359ab7b7afa0403a9ee4d6af509c6c9e (table test-table [id=fc70f8e8b05e408e976dd6c574ecf4fe]) successfully deleted
I20260812 19:07:44.799414 15665 ts_tablet_manager.cc:1905] T 359ab7b7afa0403a9ee4d6af509c6c9e P 56959f2c2e114e58b05f31f25ef3633c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 19:07:44.800513 15048 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.130.1:41731 (error 108)
I20260812 19:07:44.800679 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 15294
I20260812 19:07:44.804473 15634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:44.804133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:41731 (local address 127.0.0.1:51208)
0812 19:07:44.804286 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:44.804370 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.1:41731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 19:07:44.804776 15048 catalog_manager.cc:4584] TS 30d265e202e0489f9eb9266050e4b4fb (127.25.130.1:41731): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 359ab7b7afa0403a9ee4d6af509c6c9e: Network error: Client connection negotiation failed: client connection to 127.25.130.1:41731: connect: Connection refused (error 111)
I20260812 19:07:44.808759 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 15462
I20260812 19:07:44.815505 15634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:44.808836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:39775 (local address 127.0.0.1:37298)
0812 19:07:44.815314 (+ 6478us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:44.815394 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.2:39775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6356}
I20260812 19:07:44.819686 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 15025
2026-08-12T19:07:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1191 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:07:44 dist-test-slave-49h3 krb5kdc[15673](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:07:44 dist-test-slave-49h3 krb5kdc[15673](info): set up 2 sockets
Aug 12 19:07:44 dist-test-slave-49h3 krb5kdc[15673](info): commencing operation
krb5kdc: starting...
W20260812 19:07:45.879858 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:07:45 dist-test-slave-49h3 krb5kdc[15673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:07:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:07:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:07:46.008292 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:39825
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:37417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:39825
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:07:46.119321 15687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:46.119606 15687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:46.119647 15687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:46.123597 15687 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:07:46.123656 15687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:46.123674 15687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:46.123689 15687 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:07:46.123708 15687 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:07:46.129041 15687 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:37417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:39825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:39825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:46.130121 15687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:46.131122 15687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:46.139559 15695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:46.139637 15692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:46.139637 15693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:46.139873 15687 server_base.cc:1034] running on GCE node
I20260812 19:07:46.140461 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:46.141680 15687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:46.142868 15687 hybrid_clock.cc:648] HybridClock initialized: now 1786561666142849 us; error 35 us; skew 500 ppm
Aug 12 19:07:46 dist-test-slave-49h3 krb5kdc[15673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561666, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:07:46.145742 15687 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:07:46.146713 15687 webserver.cc:460] Webserver started at http://127.25.130.62:33219/ using document root <none> and password file <none>
I20260812 19:07:46.147280 15687 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:46.147348 15687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:46.147552 15687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:46.149760 15687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "9b98973b07a34eb794434d0516f40d78"
format_stamp: "Formatted at 2026-08-12 19:07:46 on dist-test-slave-49h3"
server_key: "33d50c0ba520c4f68dcb23c9ea8120b5"
server_key_iv: "3c19ef1862c991916a09970c28ac6169"
server_key_version: "encryptionkey@0"
I20260812 19:07:46.150338 15687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "9b98973b07a34eb794434d0516f40d78"
format_stamp: "Formatted at 2026-08-12 19:07:46 on dist-test-slave-49h3"
server_key: "33d50c0ba520c4f68dcb23c9ea8120b5"
server_key_iv: "3c19ef1862c991916a09970c28ac6169"
server_key_version: "encryptionkey@0"
I20260812 19:07:46.154220 15687 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:07:46.156764 15705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:46.158183 15687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:07:46.158319 15687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "9b98973b07a34eb794434d0516f40d78"
format_stamp: "Formatted at 2026-08-12 19:07:46 on dist-test-slave-49h3"
server_key: "33d50c0ba520c4f68dcb23c9ea8120b5"
server_key_iv: "3c19ef1862c991916a09970c28ac6169"
server_key_version: "encryptionkey@0"
I20260812 19:07:46.158438 15687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:46.175068 15687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:46.177975 15687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:46.178169 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:46.185175 15687 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:39825
I20260812 19:07:46.185216 15770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:39825 every 8 connection(s)
I20260812 19:07:46.186309 15687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:07:46.189198 15773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:46.195133 15773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b98973b07a34eb794434d0516f40d78: Bootstrap starting.
I20260812 19:07:46.195428 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 15687
I20260812 19:07:46.195566 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:07:46.195850 26120 external_mini_cluster.cc:1428] Setting key 19ff26218f0aeedca7e109e3c0ab0a9f
I20260812 19:07:46.198019 15773 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b98973b07a34eb794434d0516f40d78: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:46.198804 15773 log.cc:824] T 00000000000000000000000000000000 P 9b98973b07a34eb794434d0516f40d78: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:46.200872 15773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b98973b07a34eb794434d0516f40d78: No bootstrap required, opened a new log
Aug 12 19:07:46 dist-test-slave-49h3 krb5kdc[15673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:07:46.204769 15773 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b98973b07a34eb794434d0516f40d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b98973b07a34eb794434d0516f40d78" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39825 } }
I20260812 19:07:46.205058 15773 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b98973b07a34eb794434d0516f40d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:46.205116 15773 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b98973b07a34eb794434d0516f40d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b98973b07a34eb794434d0516f40d78, State: Initialized, Role: FOLLOWER
I20260812 19:07:46.205631 15773 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b98973b07a34eb794434d0516f40d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b98973b07a34eb794434d0516f40d78" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39825 } }
I20260812 19:07:46.205780 15773 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b98973b07a34eb794434d0516f40d78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:07:46.205853 15773 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b98973b07a34eb794434d0516f40d78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:07:46.205960 15773 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b98973b07a34eb794434d0516f40d78 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:46.206966 15773 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b98973b07a34eb794434d0516f40d78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b98973b07a34eb794434d0516f40d78" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39825 } }
I20260812 19:07:46.207306 15773 leader_election.cc:302] T 00000000000000000000000000000000 P 9b98973b07a34eb794434d0516f40d78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b98973b07a34eb794434d0516f40d78; no voters:
I20260812 19:07:46.207650 15773 leader_election.cc:288] T 00000000000000000000000000000000 P 9b98973b07a34eb794434d0516f40d78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:07:46.207773 15779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b98973b07a34eb794434d0516f40d78 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:07:46.207952 15779 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b98973b07a34eb794434d0516f40d78 [term 1 LEADER]: Becoming Leader. State: Replica: 9b98973b07a34eb794434d0516f40d78, State: Running, Role: LEADER
I20260812 19:07:46.208273 15779 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b98973b07a34eb794434d0516f40d78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b98973b07a34eb794434d0516f40d78" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39825 } }
I20260812 19:07:46.209040 15773 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b98973b07a34eb794434d0516f40d78 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:07:46.210011 15780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b98973b07a34eb794434d0516f40d78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b98973b07a34eb794434d0516f40d78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b98973b07a34eb794434d0516f40d78" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39825 } } }
I20260812 19:07:46.210131 15780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b98973b07a34eb794434d0516f40d78 [sys.catalog]: This master's current role is: LEADER
I20260812 19:07:46.210604 15776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:46.197271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47136 (local address 127.25.130.62:39825)
0812 19:07:46.197799 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:46.197816 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:46.197854 (+ 38us) server_negotiation.cc:407] Connection header received
0812 19:07:46.198583 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:46.198604 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:46.198964 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:46.199314 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:46.200133 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:46.201178 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:46.202066 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:46.202566 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:46.202717 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:46.205478 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:46.205486 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:46.205504 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:07:46.205536 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:46.207238 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:46.207706 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:46.207713 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:46.207719 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:46.207875 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:46.208223 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:46.208226 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:46.208228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:46.208696 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:46.208886 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:46.209191 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 19:07:46.209546 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":175,"threads_started":1}
I20260812 19:07:46.210644 15782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b98973b07a34eb794434d0516f40d78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b98973b07a34eb794434d0516f40d78. Latest consensus state: current_term: 1 leader_uuid: "9b98973b07a34eb794434d0516f40d78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b98973b07a34eb794434d0516f40d78" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39825 } } }
I20260812 19:07:46.210937 15782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b98973b07a34eb794434d0516f40d78 [sys.catalog]: This master's current role is: LEADER
I20260812 19:07:46.211234 15790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:07:46.214705 15790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:07:46.220067 15790 catalog_manager.cc:1347] Generated new cluster ID: f35044724925462d8a2880ca07941d42
I20260812 19:07:46.220141 15790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:07:46.238879 15790 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:07:46.239852 15790 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:07:46.252848 15790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b98973b07a34eb794434d0516f40d78: Generated new TSK 0
I20260812 19:07:46.253635 15790 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
I20260812 19:07:46.285593 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:0
--local_ip_for_outbound_sockets=127.25.130.1
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:39825
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:37417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:07:46.402529 15802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:46.402813 15802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:46.402858 15802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:46.407032 15802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:46.407119 15802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:46.407179 15802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:07:46.413023 15802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:37417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:39825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:46.414227 15802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:46.415304 15802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:46.425254 15807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:46.425240 15812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:46.425235 15808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:46.425308 15802 server_base.cc:1034] running on GCE node
I20260812 19:07:46.425940 15802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:46.426666 15802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:46.427819 15802 hybrid_clock.cc:648] HybridClock initialized: now 1786561666427788 us; error 44 us; skew 500 ppm
Aug 12 19:07:46 dist-test-slave-49h3 krb5kdc[15673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561666, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:07:46.430723 15802 init.cc:377] Logged in from keytab as kudu/127.25.130.1@KRBTEST.COM (short username kudu)
I20260812 19:07:46.431663 15802 webserver.cc:460] Webserver started at http://127.25.130.1:39063/ using document root <none> and password file <none>
I20260812 19:07:46.432210 15802 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:46.432271 15802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:46.432444 15802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:46.434525 15802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-0/data/instance:
uuid: "98417b4137674d378e5c31cb159e9c8b"
format_stamp: "Formatted at 2026-08-12 19:07:46 on dist-test-slave-49h3"
server_key: "f0e570c490c6182048cdc52631c7f695"
server_key_iv: "5487cc4eaa1af2a1b086a5c9fc41a486"
server_key_version: "encryptionkey@0"
I20260812 19:07:46.435058 15802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance:
uuid: "98417b4137674d378e5c31cb159e9c8b"
format_stamp: "Formatted at 2026-08-12 19:07:46 on dist-test-slave-49h3"
server_key: "f0e570c490c6182048cdc52631c7f695"
server_key_iv: "5487cc4eaa1af2a1b086a5c9fc41a486"
server_key_version: "encryptionkey@0"
I20260812 19:07:46.438757 15802 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 19:07:46.441218 15824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:46.442487 15802 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:07:46.442610 15802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "98417b4137674d378e5c31cb159e9c8b"
format_stamp: "Formatted at 2026-08-12 19:07:46 on dist-test-slave-49h3"
server_key: "f0e570c490c6182048cdc52631c7f695"
server_key_iv: "5487cc4eaa1af2a1b086a5c9fc41a486"
server_key_version: "encryptionkey@0"
I20260812 19:07:46.442716 15802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:46.467152 15802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:46.470235 15802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:46.470439 15802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:46.471077 15802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:07:46.471949 15802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:07:46.472000 15802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:46.472080 15802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:07:46.472107 15802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:46.480865 15802 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:45979
I20260812 19:07:46.480899 15960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:45979 every 8 connection(s)
I20260812 19:07:46.481844 15802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:07:46.483209 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 15802
I20260812 19:07:46.483353 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance
I20260812 19:07:46.483685 26120 external_mini_cluster.cc:1428] Setting key dacf5aeebaec320a62e7ef0c1beddcbf
Aug 12 19:07:46 dist-test-slave-49h3 krb5kdc[15673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561666, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.130.2@KRBTEST.COM; defaulting to no policy
I20260812 19:07:46.496305 15776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:46.483807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:54503 (local address 127.25.130.62:39825)
0812 19:07:46.483945 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:46.483949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:46.485017 (+ 1068us) server_negotiation.cc:407] Connection header received
0812 19:07:46.486541 (+ 1524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:46.486545 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:46.486601 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:46.486680 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:46.487909 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:46.488508 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:46.489771 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:46.490141 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:46.490248 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:46.492720 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:46.492724 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:46.492727 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:07:46.492760 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:46.494312 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:46.494836 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:46.494838 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:46.494839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:46.494917 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:46.495300 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:46.495302 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:46.495303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:46.495499 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:46.495581 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:46.496063 (+ 482us) server_negotiation.cc:299] Negotiation successful
0812 19:07:46.496174 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.2@KRBTEST.COM" created.
I20260812 19:07:46.497290 15963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:46.484295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:39825 (local address 127.25.130.1:54503)
0812 19:07:46.484870 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:46.484932 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:46.486328 (+ 1396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:46.486875 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:46.486882 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:46.487251 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:07:46.487750 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:46.487765 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:46.488656 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:46.488659 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:46.489637 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:46.489648 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:46.490295 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:46.490298 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:46.490451 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:46.491081 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:07:46.491097 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:07:46.492529 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:07:46.494452 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:46.494457 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:46.494472 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:46.494752 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:46.495003 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:46.495006 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:46.495008 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:46.495205 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:46.495579 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:07:46.495585 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:07:46.495848 (+ 263us) client_negotiation.cc:769] Sending connection context
0812 19:07:46.496062 (+ 214us) client_negotiation.cc:240] Negotiation successful
0812 19:07:46.496376 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":154,"threads_started":1}
I20260812 19:07:46.498656 15961 heartbeater.cc:347] Connected to a master server at 127.25.130.62:39825
I20260812 19:07:46.498901 15961 heartbeater.cc:464] Registering TS with master...
I20260812 19:07:46.499454 15961 heartbeater.cc:511] Master 127.25.130.62:39825 requested a full tablet report, sending...
I20260812 19:07:46.500995 15729 ts_manager.cc:194] Registered new tserver with Master: 98417b4137674d378e5c31cb159e9c8b (127.25.130.1:45979)
I20260812 19:07:46.502238 15729 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.1@KRBTEST.COM'} at 127.25.130.1:54503
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
I20260812 19:07:46.512424 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:0
--local_ip_for_outbound_sockets=127.25.130.2
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:39825
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:37417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:07:46.626672 15966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:46.626957 15966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:46.627022 15966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:46.631127 15966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:46.631194 15966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:46.631278 15966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:07:46.636746 15966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:37417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:39825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:46.638027 15966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:46.639196 15966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:46.648953 15971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:46.648998 15966 server_base.cc:1034] running on GCE node
W20260812 19:07:46.648957 15975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:46.648957 15972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:46.649626 15966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:46.650259 15966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:46.651415 15966 hybrid_clock.cc:648] HybridClock initialized: now 1786561666651392 us; error 34 us; skew 500 ppm
Aug 12 19:07:46 dist-test-slave-49h3 krb5kdc[15673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561666, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:07:46.654129 15966 init.cc:377] Logged in from keytab as kudu/127.25.130.2@KRBTEST.COM (short username kudu)
I20260812 19:07:46.655035 15966 webserver.cc:460] Webserver started at http://127.25.130.2:46609/ using document root <none> and password file <none>
I20260812 19:07:46.655578 15966 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:46.655640 15966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:46.655849 15966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:46.658080 15966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-1/data/instance:
uuid: "6fb5f04c92cf4cf780a92f547971bcff"
format_stamp: "Formatted at 2026-08-12 19:07:46 on dist-test-slave-49h3"
server_key: "5e7a14ed1a3e26b3fd4d42e5b875b1e1"
server_key_iv: "2c438fb8f5f9bcd5c268506f9e9bb2ac"
server_key_version: "encryptionkey@0"
I20260812 19:07:46.658656 15966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance:
uuid: "6fb5f04c92cf4cf780a92f547971bcff"
format_stamp: "Formatted at 2026-08-12 19:07:46 on dist-test-slave-49h3"
server_key: "5e7a14ed1a3e26b3fd4d42e5b875b1e1"
server_key_iv: "2c438fb8f5f9bcd5c268506f9e9bb2ac"
server_key_version: "encryptionkey@0"
I20260812 19:07:46.662910 15966 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:07:46.665701 15985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:46.667070 15966 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:07:46.667204 15966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "6fb5f04c92cf4cf780a92f547971bcff"
format_stamp: "Formatted at 2026-08-12 19:07:46 on dist-test-slave-49h3"
server_key: "5e7a14ed1a3e26b3fd4d42e5b875b1e1"
server_key_iv: "2c438fb8f5f9bcd5c268506f9e9bb2ac"
server_key_version: "encryptionkey@0"
I20260812 19:07:46.667320 15966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:46.688611 15966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:46.691751 15966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:46.691947 15966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:46.692590 15966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:07:46.693483 15966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:07:46.693542 15966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:46.693599 15966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:07:46.693624 15966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:46.702363 15966 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:37725
I20260812 19:07:46.702396 16122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:37725 every 8 connection(s)
I20260812 19:07:46.703295 15966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:07:46.710132 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 15966
I20260812 19:07:46.710275 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance
I20260812 19:07:46.710618 26120 external_mini_cluster.cc:1428] Setting key 74503ec730140c99d76768cf925f9bcb
Aug 12 19:07:46 dist-test-slave-49h3 krb5kdc[15673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561666, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:07:46.717800 15776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:46.705174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:44613 (local address 127.25.130.62:39825)
0812 19:07:46.705328 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:46.705332 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:46.706308 (+ 976us) server_negotiation.cc:407] Connection header received
0812 19:07:46.707618 (+ 1310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:46.707621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:46.707685 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:46.707759 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:46.708997 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:46.709635 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:46.711290 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:46.711580 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:46.711699 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:46.714336 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:46.714339 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:46.714341 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:07:46.714368 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:46.715695 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:46.716212 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:46.716215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:46.716216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:46.716305 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:46.716629 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:46.716632 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:46.716634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:46.716829 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:46.716919 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:46.717489 (+ 570us) server_negotiation.cc:299] Negotiation successful
0812 19:07:46.717626 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 19:07:46.718714 16125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:46.705622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:39825 (local address 127.25.130.2:44613)
0812 19:07:46.706148 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:46.706197 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:46.707388 (+ 1191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:46.707952 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:46.707959 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:46.708338 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:07:46.708840 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:46.708855 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:46.709769 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:46.709772 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:46.711156 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:46.711168 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:46.711737 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:46.711740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:46.711918 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:46.712522 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:07:46.712540 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:07:46.714159 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:07:46.715817 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:46.715824 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:46.715842 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:46.716115 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:46.716388 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:46.716392 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:46.716394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:46.716542 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:46.716929 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:07:46.716937 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:07:46.717213 (+ 276us) client_negotiation.cc:769] Sending connection context
0812 19:07:46.717399 (+ 186us) client_negotiation.cc:240] Negotiation successful
0812 19:07:46.717812 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":128,"threads_started":1}
I20260812 19:07:46.720088 16123 heartbeater.cc:347] Connected to a master server at 127.25.130.62:39825
I20260812 19:07:46.720331 16123 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.25.130.3@KRBTEST.COM; defaulting to no policy
I20260812 19:07:46.720873 16123 heartbeater.cc:511] Master 127.25.130.62:39825 requested a full tablet report, sending...
I20260812 19:07:46.722023 15729 ts_manager.cc:194] Registered new tserver with Master: 6fb5f04c92cf4cf780a92f547971bcff (127.25.130.2:37725)
I20260812 19:07:46.722635 15729 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.2@KRBTEST.COM'} at 127.25.130.2:44613
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
I20260812 19:07:46.738607 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:0
--local_ip_for_outbound_sockets=127.25.130.3
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:39825
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:37417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:07:46.852291 16128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:46.852591 16128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:46.852635 16128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:46.856822 16128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:46.856905 16128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:46.856968 16128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:07:46.862689 16128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:37417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:39825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:46.863875 16128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:46.864923 16128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:46.875016 16135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:46.875018 16134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:46.875018 16138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:46.875409 16128 server_base.cc:1034] running on GCE node
I20260812 19:07:46.875849 16128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:46.876538 16128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:46.877727 16128 hybrid_clock.cc:648] HybridClock initialized: now 1786561666877710 us; error 34 us; skew 500 ppm
Aug 12 19:07:46 dist-test-slave-49h3 krb5kdc[15673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561666, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:07:46.886162 16128 init.cc:377] Logged in from keytab as kudu/127.25.130.3@KRBTEST.COM (short username kudu)
I20260812 19:07:46.887181 16128 webserver.cc:460] Webserver started at http://127.25.130.3:37291/ using document root <none> and password file <none>
I20260812 19:07:46.887768 16128 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:46.887835 16128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:46.888047 16128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:46.890271 16128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-2/data/instance:
uuid: "50140bec814a4205868c5f49f873d4e5"
format_stamp: "Formatted at 2026-08-12 19:07:46 on dist-test-slave-49h3"
server_key: "f104528968b697840eccbc688a37b3aa"
server_key_iv: "e7f6cbbe10cbf9d2e124d2534324b128"
server_key_version: "encryptionkey@0"
I20260812 19:07:46.890851 16128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance:
uuid: "50140bec814a4205868c5f49f873d4e5"
format_stamp: "Formatted at 2026-08-12 19:07:46 on dist-test-slave-49h3"
server_key: "f104528968b697840eccbc688a37b3aa"
server_key_iv: "e7f6cbbe10cbf9d2e124d2534324b128"
server_key_version: "encryptionkey@0"
I20260812 19:07:46.894668 16128 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:07:46.897187 16145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:46.898464 16128 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:07:46.898568 16128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "50140bec814a4205868c5f49f873d4e5"
format_stamp: "Formatted at 2026-08-12 19:07:46 on dist-test-slave-49h3"
server_key: "f104528968b697840eccbc688a37b3aa"
server_key_iv: "e7f6cbbe10cbf9d2e124d2534324b128"
server_key_version: "encryptionkey@0"
I20260812 19:07:46.898681 16128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:46.914644 16128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:46.917651 16128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:46.917829 16128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:46.918460 16128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:07:46.919322 16128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:07:46.919370 16128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:46.919416 16128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:07:46.919445 16128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:46.928002 16128 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:38667
I20260812 19:07:46.928030 16288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:38667 every 8 connection(s)
I20260812 19:07:46.928948 16128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:07:46.936154 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 16128
I20260812 19:07:46.936295 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance
I20260812 19:07:46.936561 26120 external_mini_cluster.cc:1428] Setting key db2e78a3429cbdae24e69642a01d9980
Aug 12 19:07:46 dist-test-slave-49h3 krb5kdc[15673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561666, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:07:46.942973 15776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:46.930797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:41731 (local address 127.25.130.62:39825)
0812 19:07:46.930923 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:46.930926 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:46.931903 (+ 977us) server_negotiation.cc:407] Connection header received
0812 19:07:46.933186 (+ 1283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:46.933189 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:46.933250 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:46.933322 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:46.934568 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:46.935153 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:46.936310 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:46.936557 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:46.936648 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:46.939349 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:46.939352 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:46.939355 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:07:46.939382 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:46.940834 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:46.941355 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:46.941359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:46.941361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:46.941484 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:46.941833 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:46.941837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:46.941839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:46.942037 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:46.942132 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:46.942669 (+ 537us) server_negotiation.cc:299] Negotiation successful
0812 19:07:46.942805 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 19:07:46.943893 16294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:46.931215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:39825 (local address 127.25.130.3:41731)
0812 19:07:46.931735 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:46.931782 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:46.932965 (+ 1183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:46.933547 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:46.933562 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:46.933937 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:07:46.934413 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:46.934426 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:46.935281 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:46.935284 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:46.936181 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:46.936191 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:46.936719 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:46.936724 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:46.936915 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:46.937541 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:07:46.937565 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:07:46.939183 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:07:46.940961 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:46.940967 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:46.940986 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:46.941257 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:46.941585 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:46.941588 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:46.941591 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:46.941745 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:46.942147 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:07:46.942154 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:07:46.942423 (+ 269us) client_negotiation.cc:769] Sending connection context
0812 19:07:46.942609 (+ 186us) client_negotiation.cc:240] Negotiation successful
0812 19:07:46.942962 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":136,"threads_started":1}
I20260812 19:07:46.945057 16289 heartbeater.cc:347] Connected to a master server at 127.25.130.62:39825
I20260812 19:07:46.945282 16289 heartbeater.cc:464] Registering TS with master...
I20260812 19:07:46.945824 16289 heartbeater.cc:511] Master 127.25.130.62:39825 requested a full tablet report, sending...
I20260812 19:07:46.946934 15729 ts_manager.cc:194] Registered new tserver with Master: 50140bec814a4205868c5f49f873d4e5 (127.25.130.3:38667)
I20260812 19:07:46.947569 15729 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.3@KRBTEST.COM'} at 127.25.130.3:41731
I20260812 19:07:46.950464 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:07:46.959975 15776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:46.952440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47148 (local address 127.25.130.62:39825)
0812 19:07:46.952569 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:46.952573 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:46.952755 (+ 182us) server_negotiation.cc:407] Connection header received
0812 19:07:46.952870 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:46.952874 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:46.952921 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:46.952982 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:46.953763 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:46.954286 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:46.955103 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:46.955331 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:46.955419 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:46.956988 (+ 1569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:46.956992 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:46.956995 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:07:46.957023 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:46.958211 (+ 1188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:46.958704 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:46.958707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:46.958708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:46.958793 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:46.959090 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:46.959093 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:46.959095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:46.959313 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:46.959450 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:46.959709 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 19:07:46.959815 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 19:07:46.963056 15729 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:07:46.966159 15729 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:07:46.981151 16306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:46.974824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:38667 (local address 127.0.0.1:35450)
0812 19:07:46.975192 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:46.975260 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:46.975386 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:46.976887 (+ 1501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:46.976890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:46.976908 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:46.977005 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:46.977010 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:46.978516 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:46.978520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:46.979752 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:46.979763 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:46.980577 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:46.980581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:46.980692 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:46.980730 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 19:07:46.980842 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 19:07:46.980973 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":130,"thread_start_us":85,"threads_started":1}
I20260812 19:07:46.981331 16311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:46.975645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40980 (local address 127.25.130.2:37725)
0812 19:07:46.976405 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:46.976416 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:46.976438 (+ 22us) server_negotiation.cc:407] Connection header received
0812 19:07:46.976509 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:46.976514 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:46.976691 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:46.976831 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:46.977195 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:46.978011 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:46.979901 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:46.980726 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:46.980833 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:46.980969 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:46.981055 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 19:07:46.981156 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":142,"threads_started":1}
I20260812 19:07:46.981417 16307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:46.975541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:45979 (local address 127.0.0.1:42288)
0812 19:07:46.976090 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:46.976104 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:46.976189 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:46.976783 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:46.976790 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:46.976813 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:46.976966 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:46.976975 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:46.978453 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:46.978457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:46.979765 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:46.979772 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:46.981037 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:46.981041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:46.981138 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:46.981161 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 19:07:46.981229 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 19:07:46.981295 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"mutex_wait_us":71,"thread_start_us":93,"threads_started":1}
I20260812 19:07:46.981544 16308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:46.975388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35450 (local address 127.25.130.3:38667)
0812 19:07:46.976479 (+ 1091us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:46.976489 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:46.976520 (+ 31us) server_negotiation.cc:407] Connection header received
0812 19:07:46.976586 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:46.976591 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:46.976744 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:46.977154 (+ 410us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:46.977302 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:46.978364 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:46.979879 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:46.980450 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:46.980562 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:46.980727 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:46.981264 (+ 537us) server_negotiation.cc:299] Negotiation successful
0812 19:07:46.981363 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":992,"thread_start_us":145,"threads_started":1}
I20260812 19:07:46.981925 16312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:46.975753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42288 (local address 127.25.130.1:45979)
0812 19:07:46.976148 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:46.976165 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:46.976190 (+ 25us) server_negotiation.cc:407] Connection header received
0812 19:07:46.976370 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:46.976377 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:46.976605 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:46.976769 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:46.977091 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:46.978317 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:46.979901 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:46.980737 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:46.980833 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:46.980969 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:46.981699 (+ 730us) server_negotiation.cc:299] Negotiation successful
0812 19:07:46.981813 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":121,"threads_started":1}
I20260812 19:07:46.981181 16309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:46.975020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:37725 (local address 127.0.0.1:40980)
0812 19:07:46.975847 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:46.975859 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:46.975952 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:46.976988 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:46.976991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:46.977007 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:46.977088 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:46.977094 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:46.978133 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:46.978137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:46.979751 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:46.979764 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:46.980847 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:46.980851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:46.980934 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:46.980955 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 19:07:46.981026 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 19:07:46.981088 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":768,"mutex_wait_us":185,"thread_start_us":126,"threads_started":1}
I20260812 19:07:46.984344 16202 tablet_service.cc:1467] Processing CreateTablet for tablet 20b2c3a523ed49e0998f363094473aac (DEFAULT_TABLE table=test-table [id=782a4fafe6944e5dba83a0c9b048bd63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:07:46.984390 16043 tablet_service.cc:1467] Processing CreateTablet for tablet 20b2c3a523ed49e0998f363094473aac (DEFAULT_TABLE table=test-table [id=782a4fafe6944e5dba83a0c9b048bd63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:07:46.985316 16043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20b2c3a523ed49e0998f363094473aac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:46.985323 16202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20b2c3a523ed49e0998f363094473aac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:46.991475 16313 tablet_bootstrap.cc:492] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5: Bootstrap starting.
I20260812 19:07:46.991233 15877 tablet_service.cc:1467] Processing CreateTablet for tablet 20b2c3a523ed49e0998f363094473aac (DEFAULT_TABLE table=test-table [id=782a4fafe6944e5dba83a0c9b048bd63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:07:46.992390 15877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20b2c3a523ed49e0998f363094473aac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:46.993965 16313 tablet_bootstrap.cc:654] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:46.994410 16314 tablet_bootstrap.cc:492] T 20b2c3a523ed49e0998f363094473aac P 6fb5f04c92cf4cf780a92f547971bcff: Bootstrap starting.
I20260812 19:07:46.995538 16313 log.cc:824] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:46.996747 16314 tablet_bootstrap.cc:654] T 20b2c3a523ed49e0998f363094473aac P 6fb5f04c92cf4cf780a92f547971bcff: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:46.997627 16314 log.cc:824] T 20b2c3a523ed49e0998f363094473aac P 6fb5f04c92cf4cf780a92f547971bcff: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:46.998018 16313 tablet_bootstrap.cc:492] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5: No bootstrap required, opened a new log
I20260812 19:07:46.998296 16313 ts_tablet_manager.cc:1397] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.002s
I20260812 19:07:46.999641 16314 tablet_bootstrap.cc:492] T 20b2c3a523ed49e0998f363094473aac P 6fb5f04c92cf4cf780a92f547971bcff: No bootstrap required, opened a new log
I20260812 19:07:46.999688 16318 tablet_bootstrap.cc:492] T 20b2c3a523ed49e0998f363094473aac P 98417b4137674d378e5c31cb159e9c8b: Bootstrap starting.
I20260812 19:07:46.999853 16314 ts_tablet_manager.cc:1397] T 20b2c3a523ed49e0998f363094473aac P 6fb5f04c92cf4cf780a92f547971bcff: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260812 19:07:47.001765 16313 raft_consensus.cc:348] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50140bec814a4205868c5f49f873d4e5" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38667 } } peers { permanent_uuid: "6fb5f04c92cf4cf780a92f547971bcff" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37725 } } peers { permanent_uuid: "98417b4137674d378e5c31cb159e9c8b" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 45979 } }
I20260812 19:07:47.001995 16313 raft_consensus.cc:374] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:47.002044 16313 raft_consensus.cc:729] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50140bec814a4205868c5f49f873d4e5, State: Initialized, Role: FOLLOWER
I20260812 19:07:47.002120 16318 tablet_bootstrap.cc:654] T 20b2c3a523ed49e0998f363094473aac P 98417b4137674d378e5c31cb159e9c8b: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:47.002501 16313 consensus_queue.cc:260] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50140bec814a4205868c5f49f873d4e5" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38667 } } peers { permanent_uuid: "6fb5f04c92cf4cf780a92f547971bcff" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37725 } } peers { permanent_uuid: "98417b4137674d378e5c31cb159e9c8b" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 45979 } }
I20260812 19:07:47.003008 16318 log.cc:824] T 20b2c3a523ed49e0998f363094473aac P 98417b4137674d378e5c31cb159e9c8b: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:47.003159 16289 heartbeater.cc:503] Master 127.25.130.62:39825 was elected leader, sending a full tablet report...
I20260812 19:07:47.003556 16313 ts_tablet_manager.cc:1428] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260812 19:07:47.004222 16314 raft_consensus.cc:348] T 20b2c3a523ed49e0998f363094473aac P 6fb5f04c92cf4cf780a92f547971bcff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50140bec814a4205868c5f49f873d4e5" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38667 } } peers { permanent_uuid: "6fb5f04c92cf4cf780a92f547971bcff" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37725 } } peers { permanent_uuid: "98417b4137674d378e5c31cb159e9c8b" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 45979 } }
I20260812 19:07:47.004565 16314 raft_consensus.cc:374] T 20b2c3a523ed49e0998f363094473aac P 6fb5f04c92cf4cf780a92f547971bcff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:47.004647 16314 raft_consensus.cc:729] T 20b2c3a523ed49e0998f363094473aac P 6fb5f04c92cf4cf780a92f547971bcff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fb5f04c92cf4cf780a92f547971bcff, State: Initialized, Role: FOLLOWER
I20260812 19:07:47.005146 16314 consensus_queue.cc:260] T 20b2c3a523ed49e0998f363094473aac P 6fb5f04c92cf4cf780a92f547971bcff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50140bec814a4205868c5f49f873d4e5" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38667 } } peers { permanent_uuid: "6fb5f04c92cf4cf780a92f547971bcff" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37725 } } peers { permanent_uuid: "98417b4137674d378e5c31cb159e9c8b" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 45979 } }
I20260812 19:07:47.005643 16318 tablet_bootstrap.cc:492] T 20b2c3a523ed49e0998f363094473aac P 98417b4137674d378e5c31cb159e9c8b: No bootstrap required, opened a new log
I20260812 19:07:47.005867 16318 ts_tablet_manager.cc:1397] T 20b2c3a523ed49e0998f363094473aac P 98417b4137674d378e5c31cb159e9c8b: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260812 19:07:47.005879 16123 heartbeater.cc:503] Master 127.25.130.62:39825 was elected leader, sending a full tablet report...
I20260812 19:07:47.006251 16314 ts_tablet_manager.cc:1428] T 20b2c3a523ed49e0998f363094473aac P 6fb5f04c92cf4cf780a92f547971bcff: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260812 19:07:47.008493 16319 raft_consensus.cc:482] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:47.008704 16319 raft_consensus.cc:504] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50140bec814a4205868c5f49f873d4e5" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38667 } } peers { permanent_uuid: "6fb5f04c92cf4cf780a92f547971bcff" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37725 } } peers { permanent_uuid: "98417b4137674d378e5c31cb159e9c8b" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 45979 } }
I20260812 19:07:47.010090 16319 leader_election.cc:288] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fb5f04c92cf4cf780a92f547971bcff (127.25.130.2:37725), 98417b4137674d378e5c31cb159e9c8b (127.25.130.1:45979)
I20260812 19:07:47.011816 16318 raft_consensus.cc:348] T 20b2c3a523ed49e0998f363094473aac P 98417b4137674d378e5c31cb159e9c8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50140bec814a4205868c5f49f873d4e5" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38667 } } peers { permanent_uuid: "6fb5f04c92cf4cf780a92f547971bcff" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37725 } } peers { permanent_uuid: "98417b4137674d378e5c31cb159e9c8b" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 45979 } }
I20260812 19:07:47.012272 16318 raft_consensus.cc:374] T 20b2c3a523ed49e0998f363094473aac P 98417b4137674d378e5c31cb159e9c8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:47.012388 16318 raft_consensus.cc:729] T 20b2c3a523ed49e0998f363094473aac P 98417b4137674d378e5c31cb159e9c8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98417b4137674d378e5c31cb159e9c8b, State: Initialized, Role: FOLLOWER
I20260812 19:07:47.013027 16318 consensus_queue.cc:260] T 20b2c3a523ed49e0998f363094473aac P 98417b4137674d378e5c31cb159e9c8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50140bec814a4205868c5f49f873d4e5" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38667 } } peers { permanent_uuid: "6fb5f04c92cf4cf780a92f547971bcff" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37725 } } peers { permanent_uuid: "98417b4137674d378e5c31cb159e9c8b" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 45979 } }
I20260812 19:07:47.014292 16318 ts_tablet_manager.cc:1428] T 20b2c3a523ed49e0998f363094473aac P 98417b4137674d378e5c31cb159e9c8b: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260812 19:07:47.014721 15961 heartbeater.cc:503] Master 127.25.130.62:39825 was elected leader, sending a full tablet report...
I20260812 19:07:47.014806 16294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:47.010264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:37725 (local address 127.25.130.3:56479)
0812 19:07:47.010435 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:47.010451 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:47.010532 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:47.011132 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:47.011136 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:47.011155 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:47.011275 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:47.011281 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:47.012256 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:47.012259 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:47.013400 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:47.013412 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:47.014381 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:47.014385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:47.014490 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:47.014526 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 19:07:47.014593 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 19:07:47.014668 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260812 19:07:47.015245 16311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:47.010699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:56479 (local address 127.25.130.2:37725)
0812 19:07:47.010817 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:47.010822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:47.010835 (+ 13us) server_negotiation.cc:407] Connection header received
0812 19:07:47.010876 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:47.010879 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:47.010927 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:47.010997 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:47.011386 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:47.012117 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:47.013584 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:47.014152 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:47.014265 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:47.014325 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:47.015054 (+ 729us) server_negotiation.cc:299] Negotiation successful
0812 19:07:47.015134 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 19:07:47.016053 16065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20b2c3a523ed49e0998f363094473aac" candidate_uuid: "50140bec814a4205868c5f49f873d4e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fb5f04c92cf4cf780a92f547971bcff" is_pre_election: true
I20260812 19:07:47.016412 16065 raft_consensus.cc:2454] T 20b2c3a523ed49e0998f363094473aac P 6fb5f04c92cf4cf780a92f547971bcff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50140bec814a4205868c5f49f873d4e5 in term 0.
I20260812 19:07:47.016850 16312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:47.011003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:39291 (local address 127.25.130.1:45979)
0812 19:07:47.011125 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:47.011130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:47.011630 (+ 500us) server_negotiation.cc:407] Connection header received
0812 19:07:47.011799 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:47.011802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:47.011846 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:47.011889 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:47.012275 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:47.013008 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:47.015497 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:47.016185 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:47.016286 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:47.016335 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:47.016678 (+ 343us) server_negotiation.cc:299] Negotiation successful
0812 19:07:47.016750 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 19:07:47.016868 16323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:47.010497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:45979 (local address 127.25.130.3:39291)
0812 19:07:47.011547 (+ 1050us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:47.011560 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:47.011696 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:47.012026 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:47.012028 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:47.012043 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:47.012158 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:47.012164 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:47.014261 (+ 2097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:47.014264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:47.015353 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:47.015366 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:47.016452 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:47.016455 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:47.016556 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:47.016579 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 19:07:47.016655 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 19:07:47.016724 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":983,"thread_start_us":153,"threads_started":1}
I20260812 19:07:47.017657 15905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20b2c3a523ed49e0998f363094473aac" candidate_uuid: "50140bec814a4205868c5f49f873d4e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98417b4137674d378e5c31cb159e9c8b" is_pre_election: true
I20260812 19:07:47.017922 15905 raft_consensus.cc:2454] T 20b2c3a523ed49e0998f363094473aac P 98417b4137674d378e5c31cb159e9c8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50140bec814a4205868c5f49f873d4e5 in term 0.
I20260812 19:07:47.018035 16149 leader_election.cc:302] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50140bec814a4205868c5f49f873d4e5, 6fb5f04c92cf4cf780a92f547971bcff; no voters:
I20260812 19:07:47.018563 16319 raft_consensus.cc:2781] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:07:47.018661 16319 raft_consensus.cc:482] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:47.018707 16319 raft_consensus.cc:3037] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:47.019807 16319 raft_consensus.cc:504] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50140bec814a4205868c5f49f873d4e5" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38667 } } peers { permanent_uuid: "6fb5f04c92cf4cf780a92f547971bcff" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37725 } } peers { permanent_uuid: "98417b4137674d378e5c31cb159e9c8b" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 45979 } }
I20260812 19:07:47.020130 16319 leader_election.cc:288] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5 [CANDIDATE]: Term 1 election: Requested vote from peers 6fb5f04c92cf4cf780a92f547971bcff (127.25.130.2:37725), 98417b4137674d378e5c31cb159e9c8b (127.25.130.1:45979)
I20260812 19:07:47.020484 16065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20b2c3a523ed49e0998f363094473aac" candidate_uuid: "50140bec814a4205868c5f49f873d4e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fb5f04c92cf4cf780a92f547971bcff"
I20260812 19:07:47.020514 15905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20b2c3a523ed49e0998f363094473aac" candidate_uuid: "50140bec814a4205868c5f49f873d4e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98417b4137674d378e5c31cb159e9c8b"
I20260812 19:07:47.020614 15905 raft_consensus.cc:3037] T 20b2c3a523ed49e0998f363094473aac P 98417b4137674d378e5c31cb159e9c8b [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:47.020618 16065 raft_consensus.cc:3037] T 20b2c3a523ed49e0998f363094473aac P 6fb5f04c92cf4cf780a92f547971bcff [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:47.021697 15905 raft_consensus.cc:2454] T 20b2c3a523ed49e0998f363094473aac P 98417b4137674d378e5c31cb159e9c8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50140bec814a4205868c5f49f873d4e5 in term 1.
I20260812 19:07:47.021734 16065 raft_consensus.cc:2454] T 20b2c3a523ed49e0998f363094473aac P 6fb5f04c92cf4cf780a92f547971bcff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50140bec814a4205868c5f49f873d4e5 in term 1.
I20260812 19:07:47.021976 16149 leader_election.cc:302] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50140bec814a4205868c5f49f873d4e5, 98417b4137674d378e5c31cb159e9c8b; no voters:
I20260812 19:07:47.022270 16319 raft_consensus.cc:2781] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:07:47.022476 16319 raft_consensus.cc:686] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5 [term 1 LEADER]: Becoming Leader. State: Replica: 50140bec814a4205868c5f49f873d4e5, State: Running, Role: LEADER
I20260812 19:07:47.022776 16319 consensus_queue.cc:237] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50140bec814a4205868c5f49f873d4e5" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38667 } } peers { permanent_uuid: "6fb5f04c92cf4cf780a92f547971bcff" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37725 } } peers { permanent_uuid: "98417b4137674d378e5c31cb159e9c8b" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 45979 } }
I20260812 19:07:47.026657 15724 catalog_manager.cc:5495] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50140bec814a4205868c5f49f873d4e5 (127.25.130.3). New cstate: current_term: 1 leader_uuid: "50140bec814a4205868c5f49f873d4e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50140bec814a4205868c5f49f873d4e5" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fb5f04c92cf4cf780a92f547971bcff" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98417b4137674d378e5c31cb159e9c8b" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 45979 } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:47.048662 16308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:47.044100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35458 (local address 127.25.130.3:38667)
0812 19:07:47.044251 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:47.044255 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:47.044382 (+ 127us) server_negotiation.cc:407] Connection header received
0812 19:07:47.044548 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:47.044551 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:47.044629 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:47.044752 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:07:47.044978 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:47.045831 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:47.046979 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:47.047251 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:47.047355 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:47.047677 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:07:47.048222 (+ 545us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:07:47.048370 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:47.048473 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 19:07:47.048514 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 19:07:47.056840 15905 raft_consensus.cc:1261] T 20b2c3a523ed49e0998f363094473aac P 98417b4137674d378e5c31cb159e9c8b [term 1 FOLLOWER]: Refusing update from remote peer 50140bec814a4205868c5f49f873d4e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:47.056972 16065 raft_consensus.cc:1261] T 20b2c3a523ed49e0998f363094473aac P 6fb5f04c92cf4cf780a92f547971bcff [term 1 FOLLOWER]: Refusing update from remote peer 50140bec814a4205868c5f49f873d4e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:47.057685 16319 consensus_queue.cc:1035] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fb5f04c92cf4cf780a92f547971bcff" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 37725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:07:47.057943 16325 consensus_queue.cc:1035] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98417b4137674d378e5c31cb159e9c8b" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 45979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:07:47.065819 16329 mvcc.cc:205] Tried to move back new op lower bound from 7317756588255776768 to 7317756588127875072. Current Snapshot: MvccSnapshot[applied={T|T < 7317756588255776768}]
I20260812 19:07:47.069624 16332 mvcc.cc:205] Tried to move back new op lower bound from 7317756588255776768 to 7317756588127875072. Current Snapshot: MvccSnapshot[applied={T|T < 7317756588255776768}]
I20260812 19:07:47.076951 15729 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:07:47.077170 15729 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:07:47.080420 15729 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9b98973b07a34eb794434d0516f40d78: Sending DeleteTablet for 3 replicas of tablet 20b2c3a523ed49e0998f363094473aac
I20260812 19:07:47.081640 16043 tablet_service.cc:1514] Processing DeleteTablet for tablet 20b2c3a523ed49e0998f363094473aac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:07:47 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:40980
I20260812 19:07:47.081683 15877 tablet_service.cc:1514] Processing DeleteTablet for tablet 20b2c3a523ed49e0998f363094473aac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:07:47 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:42288
I20260812 19:07:47.082249 16342 tablet_replica.cc:330] T 20b2c3a523ed49e0998f363094473aac P 6fb5f04c92cf4cf780a92f547971bcff: stopping tablet replica
I20260812 19:07:47.082631 16342 raft_consensus.cc:2229] T 20b2c3a523ed49e0998f363094473aac P 6fb5f04c92cf4cf780a92f547971bcff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:47.082731 16202 tablet_service.cc:1514] Processing DeleteTablet for tablet 20b2c3a523ed49e0998f363094473aac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:07:47 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:35450
I20260812 19:07:47.083000 16342 raft_consensus.cc:2258] T 20b2c3a523ed49e0998f363094473aac P 6fb5f04c92cf4cf780a92f547971bcff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:47.083160 16343 tablet_replica.cc:330] T 20b2c3a523ed49e0998f363094473aac P 98417b4137674d378e5c31cb159e9c8b: stopping tablet replica
I20260812 19:07:47.083583 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 15802
I20260812 19:07:47.089758 16342 ts_tablet_manager.cc:1905] T 20b2c3a523ed49e0998f363094473aac P 6fb5f04c92cf4cf780a92f547971bcff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:07:47.091984 16344 tablet_replica.cc:330] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5: stopping tablet replica
I20260812 19:07:47.092120 16342 ts_tablet_manager.cc:1918] T 20b2c3a523ed49e0998f363094473aac P 6fb5f04c92cf4cf780a92f547971bcff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:07:47.092232 16342 log.cc:1196] T 20b2c3a523ed49e0998f363094473aac P 6fb5f04c92cf4cf780a92f547971bcff: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-1/wal/wals/20b2c3a523ed49e0998f363094473aac
I20260812 19:07:47.092446 16344 raft_consensus.cc:2229] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5 [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:07:47.092568 16342 ts_tablet_manager.cc:1939] T 20b2c3a523ed49e0998f363094473aac P 6fb5f04c92cf4cf780a92f547971bcff: Deleting consensus metadata
I20260812 19:07:47.092857 16344 raft_consensus.cc:2258] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:47.093961 15709 catalog_manager.cc:4841] TS 6fb5f04c92cf4cf780a92f547971bcff (127.25.130.2:37725): tablet 20b2c3a523ed49e0998f363094473aac (table test-table [id=782a4fafe6944e5dba83a0c9b048bd63]) successfully deleted
I20260812 19:07:47.094404 16344 ts_tablet_manager.cc:1905] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:07:47.097117 16344 ts_tablet_manager.cc:1918] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:07:47.097241 16344 log.cc:1196] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561415988404-26120-0/minicluster-data/ts-2/wal/wals/20b2c3a523ed49e0998f363094473aac
I20260812 19:07:47.097620 16344 ts_tablet_manager.cc:1939] T 20b2c3a523ed49e0998f363094473aac P 50140bec814a4205868c5f49f873d4e5: Deleting consensus metadata
W20260812 19:07:47.098891 15709 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.130.1:45979 (error 108)
I20260812 19:07:47.099005 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 15966
I20260812 19:07:47.101693 16309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:47.101317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:45979 (local address 127.0.0.1:42300)
0812 19:07:47.101526 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:47.101590 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.1:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260812 19:07:47.101948 15709 catalog_manager.cc:4584] TS 98417b4137674d378e5c31cb159e9c8b (127.25.130.1:45979): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 20b2c3a523ed49e0998f363094473aac: Network error: Client connection negotiation failed: client connection to 127.25.130.1:45979: connect: Connection refused (error 111)
I20260812 19:07:47.106882 15706 catalog_manager.cc:4841] TS 50140bec814a4205868c5f49f873d4e5 (127.25.130.3:38667): tablet 20b2c3a523ed49e0998f363094473aac (table test-table [id=782a4fafe6944e5dba83a0c9b048bd63]) successfully deleted
I20260812 19:07:47.111722 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 16128
I20260812 19:07:47.123761 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 15687
2026-08-12T19:07:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2305 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T19:07:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:07:47Z Disabled control of system clock
I20260812 19:07:47.159387 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:40679
--webserver_interface=127.25.130.62
--webserver_port=0
--builtin_ntp_servers=127.25.130.20:46677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:40679
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 19:07:47.272518 16351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:47.272841 16351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:47.272894 16351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:47.277024 16351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:07:47.277105 16351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:47.277128 16351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:47.277149 16351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:07:47.277169 16351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:07:47.282199 16351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:46677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:40679
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:40679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:47.283406 16351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:47.284888 16351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:47.293722 16364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:47.293725 16356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:47.293725 16358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:47.293795 16351 server_base.cc:1034] running on GCE node
I20260812 19:07:47.294595 16351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:47.295773 16351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:47.297029 16351 hybrid_clock.cc:648] HybridClock initialized: now 1786561667297011 us; error 34 us; skew 500 ppm
I20260812 19:07:47.299067 16351 webserver.cc:460] Webserver started at http://127.25.130.62:36179/ using document root <none> and password file <none>
I20260812 19:07:47.299662 16351 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:47.299738 16351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:47.299958 16351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:47.302160 16351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "9095cfd6a7534ee7a164404283eb0911"
format_stamp: "Formatted at 2026-08-12 19:07:47 on dist-test-slave-49h3"
server_key: "e3d629c9e713af854dfc3f2eeaf58f5f"
server_key_iv: "05ef837641ad509744124c903eff38e1"
server_key_version: "encryptionkey@0"
I20260812 19:07:47.302749 16351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "9095cfd6a7534ee7a164404283eb0911"
format_stamp: "Formatted at 2026-08-12 19:07:47 on dist-test-slave-49h3"
server_key: "e3d629c9e713af854dfc3f2eeaf58f5f"
server_key_iv: "05ef837641ad509744124c903eff38e1"
server_key_version: "encryptionkey@0"
I20260812 19:07:47.306577 16351 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:07:47.309046 16371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:47.310352 16351 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:07:47.310483 16351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "9095cfd6a7534ee7a164404283eb0911"
format_stamp: "Formatted at 2026-08-12 19:07:47 on dist-test-slave-49h3"
server_key: "e3d629c9e713af854dfc3f2eeaf58f5f"
server_key_iv: "05ef837641ad509744124c903eff38e1"
server_key_version: "encryptionkey@0"
I20260812 19:07:47.310593 16351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:47.328164 16351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:47.328958 16351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:47.329154 16351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:47.336351 16351 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:40679
I20260812 19:07:47.336375 16434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:40679 every 8 connection(s)
I20260812 19:07:47.337433 16351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:07:47.340761 16435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:47.346058 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 16351
I20260812 19:07:47.346202 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:07:47.346494 26120 external_mini_cluster.cc:1428] Setting key c9fc03e3cd3985af67d61504c0dfa575
I20260812 19:07:47.347023 16435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9095cfd6a7534ee7a164404283eb0911: Bootstrap starting.
I20260812 19:07:47.349750 16435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9095cfd6a7534ee7a164404283eb0911: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:47.350567 16435 log.cc:824] T 00000000000000000000000000000000 P 9095cfd6a7534ee7a164404283eb0911: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:47.352694 16435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9095cfd6a7534ee7a164404283eb0911: No bootstrap required, opened a new log
I20260812 19:07:47.356429 16438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:47.348083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57738 (local address 127.25.130.62:40679)
0812 19:07:47.348825 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:47.348840 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:47.348878 (+ 38us) server_negotiation.cc:407] Connection header received
0812 19:07:47.349634 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:47.349655 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:47.350140 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:47.350466 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:47.350910 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:47.352065 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:47.353019 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:47.353577 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:47.353742 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:47.354573 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:47.354580 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:47.354595 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:07:47.355006 (+ 411us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:47.355065 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:07:47.355097 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:47.355185 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:47.355341 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 19:07:47.355625 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":151,"threads_started":1}
I20260812 19:07:47.356472 16435 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9095cfd6a7534ee7a164404283eb0911 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9095cfd6a7534ee7a164404283eb0911" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40679 } }
I20260812 19:07:47.356808 16435 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9095cfd6a7534ee7a164404283eb0911 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:47.356884 16435 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9095cfd6a7534ee7a164404283eb0911 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9095cfd6a7534ee7a164404283eb0911, State: Initialized, Role: FOLLOWER
I20260812 19:07:47.357378 16435 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9095cfd6a7534ee7a164404283eb0911 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9095cfd6a7534ee7a164404283eb0911" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40679 } }
I20260812 19:07:47.357569 16435 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9095cfd6a7534ee7a164404283eb0911 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:07:47.357645 16435 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9095cfd6a7534ee7a164404283eb0911 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:07:47.357749 16435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9095cfd6a7534ee7a164404283eb0911 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:47.358753 16435 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9095cfd6a7534ee7a164404283eb0911 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9095cfd6a7534ee7a164404283eb0911" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40679 } }
I20260812 19:07:47.359091 16435 leader_election.cc:302] T 00000000000000000000000000000000 P 9095cfd6a7534ee7a164404283eb0911 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9095cfd6a7534ee7a164404283eb0911; no voters:
I20260812 19:07:47.359380 16435 leader_election.cc:288] T 00000000000000000000000000000000 P 9095cfd6a7534ee7a164404283eb0911 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:07:47.359481 16441 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9095cfd6a7534ee7a164404283eb0911 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:07:47.359695 16441 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9095cfd6a7534ee7a164404283eb0911 [term 1 LEADER]: Becoming Leader. State: Replica: 9095cfd6a7534ee7a164404283eb0911, State: Running, Role: LEADER
I20260812 19:07:47.360044 16441 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9095cfd6a7534ee7a164404283eb0911 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9095cfd6a7534ee7a164404283eb0911" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40679 } }
I20260812 19:07:47.360558 16435 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9095cfd6a7534ee7a164404283eb0911 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:07:47.361611 16444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9095cfd6a7534ee7a164404283eb0911 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9095cfd6a7534ee7a164404283eb0911. Latest consensus state: current_term: 1 leader_uuid: "9095cfd6a7534ee7a164404283eb0911" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9095cfd6a7534ee7a164404283eb0911" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40679 } } }
I20260812 19:07:47.361757 16444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9095cfd6a7534ee7a164404283eb0911 [sys.catalog]: This master's current role is: LEADER
I20260812 19:07:47.362142 16453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:07:47.362066 16442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9095cfd6a7534ee7a164404283eb0911 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9095cfd6a7534ee7a164404283eb0911" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9095cfd6a7534ee7a164404283eb0911" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40679 } } }
I20260812 19:07:47.362767 16442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9095cfd6a7534ee7a164404283eb0911 [sys.catalog]: This master's current role is: LEADER
I20260812 19:07:47.364939 16453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:07:47.370529 16453 catalog_manager.cc:1347] Generated new cluster ID: e70e0eb4a3e548ecbfb6dfa19d4752ea
I20260812 19:07:47.370621 16453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:07:47.381170 16453 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:07:47.382171 16453 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:07:47.387931 16453 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9095cfd6a7534ee7a164404283eb0911: Generated new TSK 0
I20260812 19:07:47.388665 16453 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:07:47.400162 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:0
--local_ip_for_outbound_sockets=127.25.130.1
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:40679
--builtin_ntp_servers=127.25.130.20:46677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 19:07:47.533877 16466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:47.534164 16466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:47.534202 16466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:47.538282 16466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:47.538344 16466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:47.538407 16466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:07:47.543265 16466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:46677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:40679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:47.544474 16466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:47.545989 16466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:47.556650 16473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:47.556650 16475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:47.556656 16471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:47.557030 16466 server_base.cc:1034] running on GCE node
I20260812 19:07:47.557492 16466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:47.558197 16466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:47.559363 16466 hybrid_clock.cc:648] HybridClock initialized: now 1786561667559346 us; error 37 us; skew 500 ppm
I20260812 19:07:47.561425 16466 webserver.cc:460] Webserver started at http://127.25.130.1:37675/ using document root <none> and password file <none>
I20260812 19:07:47.562117 16466 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:47.562202 16466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:47.562422 16466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:47.564568 16466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-0/data/instance:
uuid: "9634147c41d04e3093f2f365b805ffec"
format_stamp: "Formatted at 2026-08-12 19:07:47 on dist-test-slave-49h3"
server_key: "98b504befa4e1979650e65b4bf13ae8f"
server_key_iv: "507546307cc92ba270994d0fbe6e84df"
server_key_version: "encryptionkey@0"
I20260812 19:07:47.565152 16466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance:
uuid: "9634147c41d04e3093f2f365b805ffec"
format_stamp: "Formatted at 2026-08-12 19:07:47 on dist-test-slave-49h3"
server_key: "98b504befa4e1979650e65b4bf13ae8f"
server_key_iv: "507546307cc92ba270994d0fbe6e84df"
server_key_version: "encryptionkey@0"
I20260812 19:07:47.569075 16466 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260812 19:07:47.571625 16481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:47.572830 16466 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:07:47.572949 16466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "9634147c41d04e3093f2f365b805ffec"
format_stamp: "Formatted at 2026-08-12 19:07:47 on dist-test-slave-49h3"
server_key: "98b504befa4e1979650e65b4bf13ae8f"
server_key_iv: "507546307cc92ba270994d0fbe6e84df"
server_key_version: "encryptionkey@0"
I20260812 19:07:47.573055 16466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:47.596885 16466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:47.597707 16466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:47.597904 16466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:47.598585 16466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:07:47.599624 16466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:07:47.599687 16466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:47.599745 16466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:07:47.599773 16466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:47.608425 16466 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:34199
I20260812 19:07:47.608477 16622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:34199 every 8 connection(s)
I20260812 19:07:47.609365 16466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:07:47.618371 16438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:47.611234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:49215 (local address 127.25.130.62:40679)
0812 19:07:47.611365 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:47.611369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:47.612444 (+ 1075us) server_negotiation.cc:407] Connection header received
0812 19:07:47.613808 (+ 1364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:47.613811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:47.613861 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:47.613934 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:47.614817 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:47.615437 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:47.616561 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:47.616800 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:47.616881 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:47.617945 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:47.617948 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:47.617950 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:07:47.617964 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:47.617984 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:07:47.617992 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:47.618046 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:47.618162 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 19:07:47.618217 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 19:07:47.618626 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 16466
I20260812 19:07:47.618753 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance
I20260812 19:07:47.619017 26120 external_mini_cluster.cc:1428] Setting key b29f2e94d06433534f244f9e953984a5
I20260812 19:07:47.619338 16625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:47.611627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:40679 (local address 127.25.130.1:49215)
0812 19:07:47.612290 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:47.612335 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:47.613602 (+ 1267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:47.614085 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:47.614092 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:47.614179 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:07:47.614686 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:47.614700 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:47.615534 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:47.615537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:47.616444 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:47.616455 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:47.616921 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:47.616924 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:47.617070 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:47.617687 (+ 617us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:07:47.617689 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:07:47.617731 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:07:47.617803 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:07:47.618052 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:07:47.618057 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:07:47.618060 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 19:07:47.618149 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 19:07:47.618398 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":112,"threads_started":1}
I20260812 19:07:47.620617 16623 heartbeater.cc:347] Connected to a master server at 127.25.130.62:40679
I20260812 19:07:47.620874 16623 heartbeater.cc:464] Registering TS with master...
I20260812 19:07:47.621059 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:0
--local_ip_for_outbound_sockets=127.25.130.2
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:40679
--builtin_ntp_servers=127.25.130.20:46677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 19:07:47.621510 16623 heartbeater.cc:511] Master 127.25.130.62:40679 requested a full tablet report, sending...
I20260812 19:07:47.623214 16394 ts_manager.cc:194] Registered new tserver with Master: 9634147c41d04e3093f2f365b805ffec (127.25.130.1:34199)
I20260812 19:07:47.624586 16394 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.130.1:49215
W20260812 19:07:47.735169 16626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:47.735466 16626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:47.735524 16626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:47.739674 16626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:47.739751 16626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:47.739830 16626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:07:47.744858 16626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:46677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:40679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:47.746089 16626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:47.747571 16626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:47.757112 16633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:47.757169 16631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:47.757169 16635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:47.757282 16626 server_base.cc:1034] running on GCE node
I20260812 19:07:47.757819 16626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:47.758453 16626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:47.759645 16626 hybrid_clock.cc:648] HybridClock initialized: now 1786561667759626 us; error 30 us; skew 500 ppm
I20260812 19:07:47.765606 16626 webserver.cc:460] Webserver started at http://127.25.130.2:37463/ using document root <none> and password file <none>
I20260812 19:07:47.766237 16626 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:47.766307 16626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:47.766507 16626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:47.768702 16626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-1/data/instance:
uuid: "616dbb32ec584209b646532e3d5743b2"
format_stamp: "Formatted at 2026-08-12 19:07:47 on dist-test-slave-49h3"
server_key: "38f2495ec923bef8b117a94279c19228"
server_key_iv: "e945bc2da2ecc532ccf2f2f2955dccaf"
server_key_version: "encryptionkey@0"
I20260812 19:07:47.769284 16626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance:
uuid: "616dbb32ec584209b646532e3d5743b2"
format_stamp: "Formatted at 2026-08-12 19:07:47 on dist-test-slave-49h3"
server_key: "38f2495ec923bef8b117a94279c19228"
server_key_iv: "e945bc2da2ecc532ccf2f2f2955dccaf"
server_key_version: "encryptionkey@0"
I20260812 19:07:47.773138 16626 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260812 19:07:47.775612 16643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:47.776921 16626 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:07:47.777046 16626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "616dbb32ec584209b646532e3d5743b2"
format_stamp: "Formatted at 2026-08-12 19:07:47 on dist-test-slave-49h3"
server_key: "38f2495ec923bef8b117a94279c19228"
server_key_iv: "e945bc2da2ecc532ccf2f2f2955dccaf"
server_key_version: "encryptionkey@0"
I20260812 19:07:47.777180 16626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:47.791230 16626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:47.791989 16626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:47.792176 16626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:47.792809 16626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:07:47.793866 16626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:07:47.793917 16626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:47.793965 16626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:07:47.793995 16626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:47.802953 16626 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:44819
I20260812 19:07:47.802990 16799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:44819 every 8 connection(s)
I20260812 19:07:47.803959 16626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:07:47.809572 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 16626
I20260812 19:07:47.809711 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance
I20260812 19:07:47.809993 26120 external_mini_cluster.cc:1428] Setting key 12d86374e30994d29b3d836853ebb802
I20260812 19:07:47.812052 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:0
--local_ip_for_outbound_sockets=127.25.130.3
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:40679
--builtin_ntp_servers=127.25.130.20:46677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 19:07:47.813812 16438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:47.805910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:53465 (local address 127.25.130.62:40679)
0812 19:07:47.806049 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:47.806053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:47.807195 (+ 1142us) server_negotiation.cc:407] Connection header received
0812 19:07:47.808622 (+ 1427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:47.808625 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:47.808680 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:47.808756 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:47.809809 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:47.810394 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:47.811666 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:47.811897 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:47.811982 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:47.813314 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:47.813318 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:47.813320 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:07:47.813335 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:47.813357 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:07:47.813365 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:47.813415 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:47.813612 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 19:07:47.813677 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 19:07:47.814881 16803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:47.806359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:40679 (local address 127.25.130.2:53465)
0812 19:07:47.807039 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:47.807083 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:47.808386 (+ 1303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:47.808923 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:47.808930 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:47.809020 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:07:47.809614 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:47.809631 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:47.810519 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:47.810522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:47.811541 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:47.811554 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:47.812080 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:47.812085 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:47.812304 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:47.812955 (+ 651us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:07:47.812958 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:07:47.813019 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:07:47.813121 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:07:47.813432 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:07:47.813468 (+ 36us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:07:47.813475 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 19:07:47.813614 (+ 139us) client_negotiation.cc:240] Negotiation successful
0812 19:07:47.813958 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":115,"threads_started":1}
I20260812 19:07:47.816022 16800 heartbeater.cc:347] Connected to a master server at 127.25.130.62:40679
I20260812 19:07:47.816303 16800 heartbeater.cc:464] Registering TS with master...
I20260812 19:07:47.816990 16800 heartbeater.cc:511] Master 127.25.130.62:40679 requested a full tablet report, sending...
I20260812 19:07:47.818282 16394 ts_manager.cc:194] Registered new tserver with Master: 616dbb32ec584209b646532e3d5743b2 (127.25.130.2:44819)
I20260812 19:07:47.818905 16394 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.130.2:53465
W20260812 19:07:47.928368 16804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:47.928689 16804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:47.928736 16804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:47.933008 16804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:47.933096 16804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:47.933157 16804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:07:47.939834 16804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:46677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:40679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:47.941424 16804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:47.943302 16804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:47.953282 16812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:47.953275 16810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:47.953285 16809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:47.953969 16804 server_base.cc:1034] running on GCE node
I20260812 19:07:47.954411 16804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:47.955075 16804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:47.956243 16804 hybrid_clock.cc:648] HybridClock initialized: now 1786561667956213 us; error 42 us; skew 500 ppm
I20260812 19:07:47.958201 16804 webserver.cc:460] Webserver started at http://127.25.130.3:41399/ using document root <none> and password file <none>
I20260812 19:07:47.958772 16804 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:47.958841 16804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:47.959074 16804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:47.961356 16804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-2/data/instance:
uuid: "82f6e6fb79f745ab9e283533a08aad09"
format_stamp: "Formatted at 2026-08-12 19:07:47 on dist-test-slave-49h3"
server_key: "c58eb767c2fdd144dda715c629c7580d"
server_key_iv: "bfc40884ac1566cfc10d37ae2a16ab84"
server_key_version: "encryptionkey@0"
I20260812 19:07:47.962031 16804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance:
uuid: "82f6e6fb79f745ab9e283533a08aad09"
format_stamp: "Formatted at 2026-08-12 19:07:47 on dist-test-slave-49h3"
server_key: "c58eb767c2fdd144dda715c629c7580d"
server_key_iv: "bfc40884ac1566cfc10d37ae2a16ab84"
server_key_version: "encryptionkey@0"
I20260812 19:07:47.965917 16804 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 19:07:47.968525 16819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:47.969882 16804 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:07:47.970022 16804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "82f6e6fb79f745ab9e283533a08aad09"
format_stamp: "Formatted at 2026-08-12 19:07:47 on dist-test-slave-49h3"
server_key: "c58eb767c2fdd144dda715c629c7580d"
server_key_iv: "bfc40884ac1566cfc10d37ae2a16ab84"
server_key_version: "encryptionkey@0"
I20260812 19:07:47.970146 16804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:47.987252 16804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:47.988004 16804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:47.988200 16804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:47.988878 16804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:07:47.989928 16804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:07:47.989986 16804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:47.990064 16804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:07:47.990094 16804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:47.998564 16804 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:46385
I20260812 19:07:47.998594 16961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:46385 every 8 connection(s)
I20260812 19:07:47.999516 16804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:07:48.000295 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 16804
I20260812 19:07:48.000402 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance
I20260812 19:07:48.000718 26120 external_mini_cluster.cc:1428] Setting key efa49d4de8d7fb6ef78d3fec03ed7227
I20260812 19:07:48.008675 16438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:48.001475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:58411 (local address 127.25.130.62:40679)
0812 19:07:48.001607 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:48.001611 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:48.002974 (+ 1363us) server_negotiation.cc:407] Connection header received
0812 19:07:48.004302 (+ 1328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:48.004304 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:48.004351 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:48.004415 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:48.005294 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:48.005846 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:48.006965 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:48.007182 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:48.007255 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:48.008271 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:48.008274 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:48.008276 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:07:48.008290 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:48.008312 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:07:48.008320 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:48.008374 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:48.008485 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 19:07:48.008543 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 19:07:48.009588 16964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:48.001982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:40679 (local address 127.25.130.3:58411)
0812 19:07:48.002825 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:48.002884 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:48.004127 (+ 1243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:48.004572 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:48.004580 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:48.004663 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:07:48.005188 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:48.005202 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:48.005939 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:48.005941 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:48.006863 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:48.006873 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:48.007305 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:48.007307 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:48.007457 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:48.008011 (+ 554us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:07:48.008012 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:07:48.008060 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:07:48.008129 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:07:48.008380 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:07:48.008385 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:07:48.008389 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 19:07:48.008475 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 19:07:48.008708 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":130,"threads_started":1}
I20260812 19:07:48.010730 16962 heartbeater.cc:347] Connected to a master server at 127.25.130.62:40679
I20260812 19:07:48.010946 16962 heartbeater.cc:464] Registering TS with master...
I20260812 19:07:48.011446 16962 heartbeater.cc:511] Master 127.25.130.62:40679 requested a full tablet report, sending...
I20260812 19:07:48.012477 16394 ts_manager.cc:194] Registered new tserver with Master: 82f6e6fb79f745ab9e283533a08aad09 (127.25.130.3:46385)
I20260812 19:07:48.013074 16394 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.130.3:58411
I20260812 19:07:48.014681 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:07:48.020546 16438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:48.016698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57750 (local address 127.25.130.62:40679)
0812 19:07:48.016842 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:48.016847 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:48.016975 (+ 128us) server_negotiation.cc:407] Connection header received
0812 19:07:48.017098 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:48.017101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:48.017147 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:48.017213 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:48.017525 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:48.018043 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:48.018899 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:48.019115 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:48.019204 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:48.020042 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:48.020046 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:48.020048 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:07:48.020061 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:48.020080 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:07:48.020087 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:48.020198 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:48.020356 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 19:07:48.020396 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 19:07:48.023568 16394 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57750:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:07:48.026628 16394 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:07:48.041546 16976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:48.035458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:46385 (local address 127.0.0.1:57906)
0812 19:07:48.036149 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:48.036162 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:48.036255 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:48.037113 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:48.037116 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:48.037135 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:48.037246 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:48.037253 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:48.038818 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:48.038821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:48.039908 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:48.039916 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:48.040893 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:48.040897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:48.041097 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:48.041127 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 19:07:48.041209 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 19:07:48.041342 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"mutex_wait_us":67,"thread_start_us":87,"threads_started":1}
I20260812 19:07:48.041606 16980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:48.036844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57356 (local address 127.25.130.1:34199)
0812 19:07:48.037290 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:48.037304 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:48.037326 (+ 22us) server_negotiation.cc:407] Connection header received
0812 19:07:48.037401 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:48.037406 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:48.037606 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:48.037758 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:48.037939 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:48.038974 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:48.040208 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:48.040743 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:48.041025 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:48.041231 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:48.041286 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 19:07:48.041395 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":127,"threads_started":1}
I20260812 19:07:48.041882 16977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:48.035499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39162 (local address 127.25.130.2:44819)
0812 19:07:48.036360 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:48.036372 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:48.036397 (+ 25us) server_negotiation.cc:407] Connection header received
0812 19:07:48.036477 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:48.036483 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:48.036698 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:48.037264 (+ 566us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:48.037390 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:48.038309 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:48.039861 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:48.040722 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:48.040844 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:48.041069 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:48.041628 (+ 559us) server_negotiation.cc:299] Negotiation successful
0812 19:07:48.041736 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":748,"thread_start_us":145,"threads_started":1}
I20260812 19:07:48.041710 16975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:48.035348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:44819 (local address 127.0.0.1:39162)
0812 19:07:48.035702 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:48.035757 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:48.035890 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:48.036873 (+ 983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:48.036879 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:48.036901 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:48.037100 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:48.037109 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:48.038411 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:48.038414 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:48.039719 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:48.039730 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:48.040853 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:48.040857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:48.041357 (+ 500us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:48.041377 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 19:07:48.041438 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 19:07:48.041534 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":337,"thread_start_us":73,"threads_started":1}
I20260812 19:07:48.041546 16978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:48.036245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:34199 (local address 127.0.0.1:57356)
0812 19:07:48.036484 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:48.036496 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:48.036575 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:48.037741 (+ 1166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:48.037744 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:48.037764 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:48.037845 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:48.037850 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:48.039078 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:48.039082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:48.040093 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:48.040104 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:48.040863 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:48.040866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:48.041058 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:48.041120 (+ 62us) client_negotiation.cc:769] Sending connection context
0812 19:07:48.041233 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 19:07:48.041364 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":97,"thread_start_us":118,"threads_started":1}
I20260812 19:07:48.042522 16979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:48.036147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57906 (local address 127.25.130.3:46385)
0812 19:07:48.036668 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:48.036679 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:48.036700 (+ 21us) server_negotiation.cc:407] Connection header received
0812 19:07:48.036775 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:48.036780 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:48.036959 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:48.037401 (+ 442us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:48.037566 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:48.038698 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:48.040042 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:48.040764 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:48.040969 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:48.042218 (+ 1249us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:48.042279 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 19:07:48.042360 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":205,"threads_started":1}
I20260812 19:07:48.045341 16538 tablet_service.cc:1467] Processing CreateTablet for tablet 17173110ac724dbc99a6b96b2f7906de (DEFAULT_TABLE table=test-table [id=b452eb51154b4bffb8a415eedce09908]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:07:48.045324 16879 tablet_service.cc:1467] Processing CreateTablet for tablet 17173110ac724dbc99a6b96b2f7906de (DEFAULT_TABLE table=test-table [id=b452eb51154b4bffb8a415eedce09908]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:07:48.045324 16708 tablet_service.cc:1467] Processing CreateTablet for tablet 17173110ac724dbc99a6b96b2f7906de (DEFAULT_TABLE table=test-table [id=b452eb51154b4bffb8a415eedce09908]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:07:48.046430 16538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17173110ac724dbc99a6b96b2f7906de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:48.046448 16708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17173110ac724dbc99a6b96b2f7906de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:48.046448 16879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17173110ac724dbc99a6b96b2f7906de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:48.053285 16981 tablet_bootstrap.cc:492] T 17173110ac724dbc99a6b96b2f7906de P 9634147c41d04e3093f2f365b805ffec: Bootstrap starting.
I20260812 19:07:48.054397 16983 tablet_bootstrap.cc:492] T 17173110ac724dbc99a6b96b2f7906de P 82f6e6fb79f745ab9e283533a08aad09: Bootstrap starting.
I20260812 19:07:48.055547 16981 tablet_bootstrap.cc:654] T 17173110ac724dbc99a6b96b2f7906de P 9634147c41d04e3093f2f365b805ffec: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:48.056069 16983 tablet_bootstrap.cc:654] T 17173110ac724dbc99a6b96b2f7906de P 82f6e6fb79f745ab9e283533a08aad09: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:48.056286 16981 log.cc:824] T 17173110ac724dbc99a6b96b2f7906de P 9634147c41d04e3093f2f365b805ffec: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:48.056739 16983 log.cc:824] T 17173110ac724dbc99a6b96b2f7906de P 82f6e6fb79f745ab9e283533a08aad09: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:48.057411 16982 tablet_bootstrap.cc:492] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2: Bootstrap starting.
I20260812 19:07:48.058629 16983 tablet_bootstrap.cc:492] T 17173110ac724dbc99a6b96b2f7906de P 82f6e6fb79f745ab9e283533a08aad09: No bootstrap required, opened a new log
I20260812 19:07:48.058631 16981 tablet_bootstrap.cc:492] T 17173110ac724dbc99a6b96b2f7906de P 9634147c41d04e3093f2f365b805ffec: No bootstrap required, opened a new log
I20260812 19:07:48.058841 16981 ts_tablet_manager.cc:1397] T 17173110ac724dbc99a6b96b2f7906de P 9634147c41d04e3093f2f365b805ffec: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 19:07:48.058856 16983 ts_tablet_manager.cc:1397] T 17173110ac724dbc99a6b96b2f7906de P 82f6e6fb79f745ab9e283533a08aad09: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 19:07:48.059422 16982 tablet_bootstrap.cc:654] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:48.060122 16982 log.cc:824] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:48.061928 16982 tablet_bootstrap.cc:492] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2: No bootstrap required, opened a new log
I20260812 19:07:48.062161 16982 ts_tablet_manager.cc:1397] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260812 19:07:48.063248 16983 raft_consensus.cc:348] T 17173110ac724dbc99a6b96b2f7906de P 82f6e6fb79f745ab9e283533a08aad09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "616dbb32ec584209b646532e3d5743b2" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 44819 } } peers { permanent_uuid: "82f6e6fb79f745ab9e283533a08aad09" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46385 } } peers { permanent_uuid: "9634147c41d04e3093f2f365b805ffec" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 34199 } }
I20260812 19:07:48.063248 16981 raft_consensus.cc:348] T 17173110ac724dbc99a6b96b2f7906de P 9634147c41d04e3093f2f365b805ffec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "616dbb32ec584209b646532e3d5743b2" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 44819 } } peers { permanent_uuid: "82f6e6fb79f745ab9e283533a08aad09" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46385 } } peers { permanent_uuid: "9634147c41d04e3093f2f365b805ffec" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 34199 } }
I20260812 19:07:48.063540 16983 raft_consensus.cc:374] T 17173110ac724dbc99a6b96b2f7906de P 82f6e6fb79f745ab9e283533a08aad09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:48.063548 16981 raft_consensus.cc:374] T 17173110ac724dbc99a6b96b2f7906de P 9634147c41d04e3093f2f365b805ffec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:48.063602 16981 raft_consensus.cc:729] T 17173110ac724dbc99a6b96b2f7906de P 9634147c41d04e3093f2f365b805ffec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9634147c41d04e3093f2f365b805ffec, State: Initialized, Role: FOLLOWER
I20260812 19:07:48.063639 16983 raft_consensus.cc:729] T 17173110ac724dbc99a6b96b2f7906de P 82f6e6fb79f745ab9e283533a08aad09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82f6e6fb79f745ab9e283533a08aad09, State: Initialized, Role: FOLLOWER
I20260812 19:07:48.064147 16981 consensus_queue.cc:260] T 17173110ac724dbc99a6b96b2f7906de P 9634147c41d04e3093f2f365b805ffec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "616dbb32ec584209b646532e3d5743b2" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 44819 } } peers { permanent_uuid: "82f6e6fb79f745ab9e283533a08aad09" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46385 } } peers { permanent_uuid: "9634147c41d04e3093f2f365b805ffec" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 34199 } }
I20260812 19:07:48.064146 16983 consensus_queue.cc:260] T 17173110ac724dbc99a6b96b2f7906de P 82f6e6fb79f745ab9e283533a08aad09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "616dbb32ec584209b646532e3d5743b2" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 44819 } } peers { permanent_uuid: "82f6e6fb79f745ab9e283533a08aad09" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46385 } } peers { permanent_uuid: "9634147c41d04e3093f2f365b805ffec" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 34199 } }
I20260812 19:07:48.065220 16983 ts_tablet_manager.cc:1428] T 17173110ac724dbc99a6b96b2f7906de P 82f6e6fb79f745ab9e283533a08aad09: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 19:07:48.065220 16981 ts_tablet_manager.cc:1428] T 17173110ac724dbc99a6b96b2f7906de P 9634147c41d04e3093f2f365b805ffec: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 19:07:48.065624 16962 heartbeater.cc:503] Master 127.25.130.62:40679 was elected leader, sending a full tablet report...
I20260812 19:07:48.065836 16623 heartbeater.cc:503] Master 127.25.130.62:40679 was elected leader, sending a full tablet report...
I20260812 19:07:48.065927 16982 raft_consensus.cc:348] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "616dbb32ec584209b646532e3d5743b2" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 44819 } } peers { permanent_uuid: "82f6e6fb79f745ab9e283533a08aad09" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46385 } } peers { permanent_uuid: "9634147c41d04e3093f2f365b805ffec" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 34199 } }
I20260812 19:07:48.066188 16982 raft_consensus.cc:374] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:48.066248 16982 raft_consensus.cc:729] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 616dbb32ec584209b646532e3d5743b2, State: Initialized, Role: FOLLOWER
I20260812 19:07:48.066658 16982 consensus_queue.cc:260] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "616dbb32ec584209b646532e3d5743b2" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 44819 } } peers { permanent_uuid: "82f6e6fb79f745ab9e283533a08aad09" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46385 } } peers { permanent_uuid: "9634147c41d04e3093f2f365b805ffec" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 34199 } }
I20260812 19:07:48.067385 16800 heartbeater.cc:503] Master 127.25.130.62:40679 was elected leader, sending a full tablet report...
I20260812 19:07:48.067728 16982 ts_tablet_manager.cc:1428] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260812 19:07:48.110652 16624 tablet.cc:2369] T 17173110ac724dbc99a6b96b2f7906de P 9634147c41d04e3093f2f365b805ffec: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 19:07:48.250568 16963 tablet.cc:2369] T 17173110ac724dbc99a6b96b2f7906de P 82f6e6fb79f745ab9e283533a08aad09: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 19:07:48.305182 16802 tablet.cc:2369] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:07:48.355721 16989 raft_consensus.cc:482] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:48.355954 16989 raft_consensus.cc:504] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "616dbb32ec584209b646532e3d5743b2" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 44819 } } peers { permanent_uuid: "82f6e6fb79f745ab9e283533a08aad09" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46385 } } peers { permanent_uuid: "9634147c41d04e3093f2f365b805ffec" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 34199 } }
I20260812 19:07:48.357146 16989 leader_election.cc:288] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 82f6e6fb79f745ab9e283533a08aad09 (127.25.130.3:46385), 9634147c41d04e3093f2f365b805ffec (127.25.130.1:34199)
I20260812 19:07:48.361176 16980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:48.357562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:60811 (local address 127.25.130.1:34199)
0812 19:07:48.357694 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:48.357698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:48.357903 (+ 205us) server_negotiation.cc:407] Connection header received
0812 19:07:48.358022 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:48.358024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:48.358066 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:48.358127 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:48.358436 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:48.358945 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:48.360096 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:48.360487 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:48.360599 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:48.360659 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:48.360946 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 19:07:48.361030 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 19:07:48.361189 16979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:48.357523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:56541 (local address 127.25.130.3:46385)
0812 19:07:48.357663 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:48.357667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:48.357825 (+ 158us) server_negotiation.cc:407] Connection header received
0812 19:07:48.357976 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:48.357979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:48.358019 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:48.358080 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:48.358381 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:48.358886 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:48.360081 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:48.360494 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:48.360608 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:48.360659 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:48.360970 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 19:07:48.361055 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 19:07:48.361228 16991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:48.357440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:34199 (local address 127.25.130.2:60811)
0812 19:07:48.357811 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:48.357822 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:48.357920 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:48.358147 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:48.358150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:48.358165 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:48.358325 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:48.358331 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:48.359048 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:48.359052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:48.359972 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:48.359983 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:48.360599 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:48.360603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:48.360799 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:48.360842 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 19:07:48.360910 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 19:07:48.361031 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":126,"thread_start_us":80,"threads_started":1}
I20260812 19:07:48.361227 16990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:48.357399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:46385 (local address 127.25.130.2:56541)
0812 19:07:48.357741 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:48.357753 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:48.357863 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:48.358100 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:48.358103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:48.358117 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:48.358280 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:48.358286 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:48.358985 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:48.358989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:48.359960 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:48.359971 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:48.360608 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:48.360612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:48.360850 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:48.360873 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 19:07:48.360939 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 19:07:48.361060 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":155,"thread_start_us":98,"threads_started":1}
I20260812 19:07:48.361950 16568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17173110ac724dbc99a6b96b2f7906de" candidate_uuid: "616dbb32ec584209b646532e3d5743b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9634147c41d04e3093f2f365b805ffec" is_pre_election: true
I20260812 19:07:48.362104 16903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17173110ac724dbc99a6b96b2f7906de" candidate_uuid: "616dbb32ec584209b646532e3d5743b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82f6e6fb79f745ab9e283533a08aad09" is_pre_election: true
I20260812 19:07:48.362293 16568 raft_consensus.cc:2454] T 17173110ac724dbc99a6b96b2f7906de P 9634147c41d04e3093f2f365b805ffec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 616dbb32ec584209b646532e3d5743b2 in term 0.
I20260812 19:07:48.362370 16903 raft_consensus.cc:2454] T 17173110ac724dbc99a6b96b2f7906de P 82f6e6fb79f745ab9e283533a08aad09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 616dbb32ec584209b646532e3d5743b2 in term 0.
I20260812 19:07:48.362814 16644 leader_election.cc:302] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 616dbb32ec584209b646532e3d5743b2, 9634147c41d04e3093f2f365b805ffec; no voters:
I20260812 19:07:48.363101 16989 raft_consensus.cc:2781] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:07:48.363194 16989 raft_consensus.cc:482] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:48.363255 16989 raft_consensus.cc:3037] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:48.364296 16989 raft_consensus.cc:504] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "616dbb32ec584209b646532e3d5743b2" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 44819 } } peers { permanent_uuid: "82f6e6fb79f745ab9e283533a08aad09" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46385 } } peers { permanent_uuid: "9634147c41d04e3093f2f365b805ffec" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 34199 } }
I20260812 19:07:48.364630 16989 leader_election.cc:288] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2 [CANDIDATE]: Term 1 election: Requested vote from peers 82f6e6fb79f745ab9e283533a08aad09 (127.25.130.3:46385), 9634147c41d04e3093f2f365b805ffec (127.25.130.1:34199)
I20260812 19:07:48.365031 16568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17173110ac724dbc99a6b96b2f7906de" candidate_uuid: "616dbb32ec584209b646532e3d5743b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9634147c41d04e3093f2f365b805ffec"
I20260812 19:07:48.365031 16903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17173110ac724dbc99a6b96b2f7906de" candidate_uuid: "616dbb32ec584209b646532e3d5743b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82f6e6fb79f745ab9e283533a08aad09"
I20260812 19:07:48.365175 16903 raft_consensus.cc:3037] T 17173110ac724dbc99a6b96b2f7906de P 82f6e6fb79f745ab9e283533a08aad09 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:48.365175 16568 raft_consensus.cc:3037] T 17173110ac724dbc99a6b96b2f7906de P 9634147c41d04e3093f2f365b805ffec [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:48.366528 16903 raft_consensus.cc:2454] T 17173110ac724dbc99a6b96b2f7906de P 82f6e6fb79f745ab9e283533a08aad09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 616dbb32ec584209b646532e3d5743b2 in term 1.
I20260812 19:07:48.366528 16568 raft_consensus.cc:2454] T 17173110ac724dbc99a6b96b2f7906de P 9634147c41d04e3093f2f365b805ffec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 616dbb32ec584209b646532e3d5743b2 in term 1.
I20260812 19:07:48.366849 16644 leader_election.cc:302] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 616dbb32ec584209b646532e3d5743b2, 9634147c41d04e3093f2f365b805ffec; no voters:
I20260812 19:07:48.367049 16989 raft_consensus.cc:2781] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:07:48.367278 16989 raft_consensus.cc:686] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2 [term 1 LEADER]: Becoming Leader. State: Replica: 616dbb32ec584209b646532e3d5743b2, State: Running, Role: LEADER
I20260812 19:07:48.367599 16989 consensus_queue.cc:237] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "616dbb32ec584209b646532e3d5743b2" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 44819 } } peers { permanent_uuid: "82f6e6fb79f745ab9e283533a08aad09" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46385 } } peers { permanent_uuid: "9634147c41d04e3093f2f365b805ffec" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 34199 } }
I20260812 19:07:48.370796 16393 catalog_manager.cc:5495] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 616dbb32ec584209b646532e3d5743b2 (127.25.130.2). New cstate: current_term: 1 leader_uuid: "616dbb32ec584209b646532e3d5743b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "616dbb32ec584209b646532e3d5743b2" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 44819 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "82f6e6fb79f745ab9e283533a08aad09" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9634147c41d04e3093f2f365b805ffec" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 34199 } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:48.406723 16977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:48.403208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39164 (local address 127.25.130.2:44819)
0812 19:07:48.403358 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:48.403363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:48.403375 (+ 12us) server_negotiation.cc:407] Connection header received
0812 19:07:48.403446 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:48.403448 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:48.403506 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:48.403603 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:07:48.403859 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:48.404354 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:48.405176 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:48.405392 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:48.405508 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:48.405845 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:07:48.406279 (+ 434us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:07:48.406414 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:48.406538 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 19:07:48.406578 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 19:07:48.413009 16568 raft_consensus.cc:1261] T 17173110ac724dbc99a6b96b2f7906de P 9634147c41d04e3093f2f365b805ffec [term 1 FOLLOWER]: Refusing update from remote peer 616dbb32ec584209b646532e3d5743b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:48.413009 16903 raft_consensus.cc:1261] T 17173110ac724dbc99a6b96b2f7906de P 82f6e6fb79f745ab9e283533a08aad09 [term 1 FOLLOWER]: Refusing update from remote peer 616dbb32ec584209b646532e3d5743b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:48.413795 16992 consensus_queue.cc:1035] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82f6e6fb79f745ab9e283533a08aad09" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 46385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:07:48.413993 16989 consensus_queue.cc:1035] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9634147c41d04e3093f2f365b805ffec" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 34199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:07:48.422607 16996 mvcc.cc:205] Tried to move back new op lower bound from 7317756593811521536 to 7317756593636397056. Current Snapshot: MvccSnapshot[applied={T|T < 7317756593811521536}]
I20260812 19:07:48.423159 16998 mvcc.cc:205] Tried to move back new op lower bound from 7317756593811521536 to 7317756593636397056. Current Snapshot: MvccSnapshot[applied={T|T < 7317756593811521536}]
I20260812 19:07:48.424358 16997 mvcc.cc:205] Tried to move back new op lower bound from 7317756593811521536 to 7317756593636397056. Current Snapshot: MvccSnapshot[applied={T|T < 7317756593811521536}]
I20260812 19:07:48.433393 16393 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:07:48.433619 16393 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:07:48.436192 16393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9095cfd6a7534ee7a164404283eb0911: Sending DeleteTablet for 3 replicas of tablet 17173110ac724dbc99a6b96b2f7906de
I20260812 19:07:48.437126 16708 tablet_service.cc:1514] Processing DeleteTablet for tablet 17173110ac724dbc99a6b96b2f7906de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:07:48 UTC) from {username='slave'} at 127.0.0.1:39162
I20260812 19:07:48.437153 16879 tablet_service.cc:1514] Processing DeleteTablet for tablet 17173110ac724dbc99a6b96b2f7906de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:07:48 UTC) from {username='slave'} at 127.0.0.1:57906
I20260812 19:07:48.437219 16538 tablet_service.cc:1514] Processing DeleteTablet for tablet 17173110ac724dbc99a6b96b2f7906de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:07:48 UTC) from {username='slave'} at 127.0.0.1:57356
I20260812 19:07:48.437765 17008 tablet_replica.cc:330] T 17173110ac724dbc99a6b96b2f7906de P 82f6e6fb79f745ab9e283533a08aad09: stopping tablet replica
I20260812 19:07:48.437776 17009 tablet_replica.cc:330] T 17173110ac724dbc99a6b96b2f7906de P 9634147c41d04e3093f2f365b805ffec: stopping tablet replica
I20260812 19:07:48.438146 17009 raft_consensus.cc:2229] T 17173110ac724dbc99a6b96b2f7906de P 9634147c41d04e3093f2f365b805ffec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:48.438146 17008 raft_consensus.cc:2229] T 17173110ac724dbc99a6b96b2f7906de P 82f6e6fb79f745ab9e283533a08aad09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:48.438526 17009 raft_consensus.cc:2258] T 17173110ac724dbc99a6b96b2f7906de P 9634147c41d04e3093f2f365b805ffec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:48.438526 17008 raft_consensus.cc:2258] T 17173110ac724dbc99a6b96b2f7906de P 82f6e6fb79f745ab9e283533a08aad09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:48.438551 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 16466
I20260812 19:07:48.442508 17010 tablet_replica.cc:330] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2: stopping tablet replica
I20260812 19:07:48.443014 17010 raft_consensus.cc:2229] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2 [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:07:48.443521 17010 raft_consensus.cc:2258] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:48.445081 17008 ts_tablet_manager.cc:1905] T 17173110ac724dbc99a6b96b2f7906de P 82f6e6fb79f745ab9e283533a08aad09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:07:48.448408 17008 ts_tablet_manager.cc:1918] T 17173110ac724dbc99a6b96b2f7906de P 82f6e6fb79f745ab9e283533a08aad09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:07:48.448546 17008 log.cc:1196] T 17173110ac724dbc99a6b96b2f7906de P 82f6e6fb79f745ab9e283533a08aad09: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561415988404-26120-0/minicluster-data/ts-2/wal/wals/17173110ac724dbc99a6b96b2f7906de
I20260812 19:07:48.448952 17008 ts_tablet_manager.cc:1939] T 17173110ac724dbc99a6b96b2f7906de P 82f6e6fb79f745ab9e283533a08aad09: Deleting consensus metadata
I20260812 19:07:48.450045 17010 ts_tablet_manager.cc:1905] T 17173110ac724dbc99a6b96b2f7906de P 616dbb32ec584209b646532e3d5743b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:07:48.450553 16375 catalog_manager.cc:4841] TS 82f6e6fb79f745ab9e283533a08aad09 (127.25.130.3:46385): tablet 17173110ac724dbc99a6b96b2f7906de (table test-table [id=b452eb51154b4bffb8a415eedce09908]) successfully deleted
W20260812 19:07:48.451377 16372 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.130.1:34199 (error 108)
I20260812 19:07:48.452492 16975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:48.452190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:34199 (local address 127.0.0.1:57364)
0812 19:07:48.452325 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:48.452397 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.1:34199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 19:07:48.452618 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 16626
W20260812 19:07:48.454301 16372 catalog_manager.cc:4584] TS 9634147c41d04e3093f2f365b805ffec (127.25.130.1:34199): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 17173110ac724dbc99a6b96b2f7906de: Network error: Client connection negotiation failed: client connection to 127.25.130.1:34199: connect: Connection refused (error 111)
I20260812 19:07:48.461700 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 16804
I20260812 19:07:48.467430 16975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:48.461888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:44819 (local address 127.0.0.1:39174)
0812 19:07:48.467222 (+ 5334us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:48.467318 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.2:44819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5189}
I20260812 19:07:48.472867 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 16351
2026-08-12T19:07:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1347 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:07:48 dist-test-slave-49h3 krb5kdc[17015](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:07:48 dist-test-slave-49h3 krb5kdc[17015](info): set up 2 sockets
Aug 12 19:07:48 dist-test-slave-49h3 krb5kdc[17015](info): commencing operation
krb5kdc: starting...
W20260812 19:07:49.528780 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:07:49 dist-test-slave-49h3 krb5kdc[17015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:07:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:07:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:07:49.656692 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:39425
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:44313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:39425
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:07:49.769321 17030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:49.769672 17030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:49.769723 17030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:49.773785 17030 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:07:49.773842 17030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:49.773861 17030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:49.773877 17030 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:07:49.773891 17030 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:07:49.779268 17030 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:44313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:39425
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:39425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:49.780324 17030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:49.781314 17030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:49.789706 17036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:49.789690 17039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:49.789690 17037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:49.789752 17030 server_base.cc:1034] running on GCE node
I20260812 19:07:49.790596 17030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:49.791728 17030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:49.792868 17030 hybrid_clock.cc:648] HybridClock initialized: now 1786561669792846 us; error 31 us; skew 500 ppm
Aug 12 19:07:49 dist-test-slave-49h3 krb5kdc[17015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561669, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:07:49.795549 17030 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:07:49.796415 17030 webserver.cc:460] Webserver started at http://127.25.130.62:36465/ using document root <none> and password file <none>
I20260812 19:07:49.796973 17030 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:49.797034 17030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:49.797245 17030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:49.799327 17030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "88ee3383b9234d0eaf68e77e781f54ed"
format_stamp: "Formatted at 2026-08-12 19:07:49 on dist-test-slave-49h3"
server_key: "acc68f4f033c196a3fb0e6802db13765"
server_key_iv: "78633d5d07042b526c21c71a0bb9fdb4"
server_key_version: "encryptionkey@0"
I20260812 19:07:49.799881 17030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "88ee3383b9234d0eaf68e77e781f54ed"
format_stamp: "Formatted at 2026-08-12 19:07:49 on dist-test-slave-49h3"
server_key: "acc68f4f033c196a3fb0e6802db13765"
server_key_iv: "78633d5d07042b526c21c71a0bb9fdb4"
server_key_version: "encryptionkey@0"
I20260812 19:07:49.803601 17030 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.001s
I20260812 19:07:49.805910 17047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:49.807180 17030 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:07:49.807309 17030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "88ee3383b9234d0eaf68e77e781f54ed"
format_stamp: "Formatted at 2026-08-12 19:07:49 on dist-test-slave-49h3"
server_key: "acc68f4f033c196a3fb0e6802db13765"
server_key_iv: "78633d5d07042b526c21c71a0bb9fdb4"
server_key_version: "encryptionkey@0"
I20260812 19:07:49.807412 17030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:49.823690 17030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:49.826884 17030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:49.827080 17030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:49.834061 17030 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:39425
I20260812 19:07:49.834090 17114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:39425 every 8 connection(s)
I20260812 19:07:49.835027 17030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:07:49.837790 17116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:49.843603 17116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88ee3383b9234d0eaf68e77e781f54ed: Bootstrap starting.
I20260812 19:07:49.843866 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 17030
I20260812 19:07:49.843987 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:07:49.844255 26120 external_mini_cluster.cc:1428] Setting key 86eca56529163340159accaa079b1d4f
I20260812 19:07:49.846381 17116 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88ee3383b9234d0eaf68e77e781f54ed: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:49.847321 17116 log.cc:824] T 00000000000000000000000000000000 P 88ee3383b9234d0eaf68e77e781f54ed: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:49.849731 17116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88ee3383b9234d0eaf68e77e781f54ed: No bootstrap required, opened a new log
Aug 12 19:07:49 dist-test-slave-49h3 krb5kdc[17015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:07:49.854120 17116 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88ee3383b9234d0eaf68e77e781f54ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88ee3383b9234d0eaf68e77e781f54ed" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39425 } }
I20260812 19:07:49.854380 17116 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88ee3383b9234d0eaf68e77e781f54ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:49.854439 17116 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88ee3383b9234d0eaf68e77e781f54ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ee3383b9234d0eaf68e77e781f54ed, State: Initialized, Role: FOLLOWER
I20260812 19:07:49.854919 17116 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88ee3383b9234d0eaf68e77e781f54ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88ee3383b9234d0eaf68e77e781f54ed" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39425 } }
I20260812 19:07:49.855088 17116 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88ee3383b9234d0eaf68e77e781f54ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:07:49.855161 17116 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88ee3383b9234d0eaf68e77e781f54ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:07:49.855278 17116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88ee3383b9234d0eaf68e77e781f54ed [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:49.856348 17116 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88ee3383b9234d0eaf68e77e781f54ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88ee3383b9234d0eaf68e77e781f54ed" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39425 } }
I20260812 19:07:49.856714 17116 leader_election.cc:302] T 00000000000000000000000000000000 P 88ee3383b9234d0eaf68e77e781f54ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88ee3383b9234d0eaf68e77e781f54ed; no voters:
I20260812 19:07:49.857072 17116 leader_election.cc:288] T 00000000000000000000000000000000 P 88ee3383b9234d0eaf68e77e781f54ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:07:49.857165 17122 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88ee3383b9234d0eaf68e77e781f54ed [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:07:49.857342 17122 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88ee3383b9234d0eaf68e77e781f54ed [term 1 LEADER]: Becoming Leader. State: Replica: 88ee3383b9234d0eaf68e77e781f54ed, State: Running, Role: LEADER
I20260812 19:07:49.857688 17122 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88ee3383b9234d0eaf68e77e781f54ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88ee3383b9234d0eaf68e77e781f54ed" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39425 } }
I20260812 19:07:49.858510 17116 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88ee3383b9234d0eaf68e77e781f54ed [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:07:49.859745 17123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88ee3383b9234d0eaf68e77e781f54ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88ee3383b9234d0eaf68e77e781f54ed. Latest consensus state: current_term: 1 leader_uuid: "88ee3383b9234d0eaf68e77e781f54ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88ee3383b9234d0eaf68e77e781f54ed" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39425 } } }
I20260812 19:07:49.859750 17124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88ee3383b9234d0eaf68e77e781f54ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88ee3383b9234d0eaf68e77e781f54ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88ee3383b9234d0eaf68e77e781f54ed" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 39425 } } }
I20260812 19:07:49.859906 17123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88ee3383b9234d0eaf68e77e781f54ed [sys.catalog]: This master's current role is: LEADER
I20260812 19:07:49.859915 17124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88ee3383b9234d0eaf68e77e781f54ed [sys.catalog]: This master's current role is: LEADER
I20260812 19:07:49.860654 17132 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:07:49.860698 17120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:49.845701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54304 (local address 127.25.130.62:39425)
0812 19:07:49.846172 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:49.846192 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:49.846236 (+ 44us) server_negotiation.cc:407] Connection header received
0812 19:07:49.847130 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:49.847155 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:49.847600 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:49.847943 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:49.848730 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:49.850202 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:49.851049 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:49.851766 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:49.851935 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:49.854659 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:49.854665 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:49.854680 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:07:49.854714 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:49.856714 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:49.857223 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:49.857230 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:49.857236 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:49.857406 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:49.857815 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:49.857818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:49.857820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:49.858293 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:49.858498 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:49.859189 (+ 691us) server_negotiation.cc:299] Negotiation successful
0812 19:07:49.859531 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":133,"threads_started":1}
I20260812 19:07:49.863917 17132 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:07:49.870108 17132 catalog_manager.cc:1347] Generated new cluster ID: 9fc1b3d48e0340db9aa65fa21f9ce698
I20260812 19:07:49.870214 17132 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:07:49.881042 17132 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:07:49.882319 17132 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:07:49.891399 17132 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88ee3383b9234d0eaf68e77e781f54ed: Generated new TSK 0
I20260812 19:07:49.892179 17132 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
Entry for principal kudu/127.25.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab.
I20260812 19:07:49.926172 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:0
--local_ip_for_outbound_sockets=127.25.130.1
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:39425
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:44313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:07:50.045130 17145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:50.045432 17145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:50.045521 17145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:50.049690 17145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:50.049765 17145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:50.049825 17145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:07:50.055446 17145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:44313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.1.keytab
--principal=kudu/127.25.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:39425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:50.056557 17145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:50.057649 17145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:50.067584 17152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:50.067605 17150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:50.067762 17145 server_base.cc:1034] running on GCE node
W20260812 19:07:50.067902 17154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:50.068375 17145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:50.069082 17145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:50.070247 17145 hybrid_clock.cc:648] HybridClock initialized: now 1786561670070224 us; error 35 us; skew 500 ppm
Aug 12 19:07:50 dist-test-slave-49h3 krb5kdc[17015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561670, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:07:50.073025 17145 init.cc:377] Logged in from keytab as kudu/127.25.130.1@KRBTEST.COM (short username kudu)
I20260812 19:07:50.073994 17145 webserver.cc:460] Webserver started at http://127.25.130.1:45969/ using document root <none> and password file <none>
I20260812 19:07:50.074544 17145 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:50.074612 17145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:50.074827 17145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:50.077085 17145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-0/data/instance:
uuid: "80ad327fdca74735873497aacc657e5e"
format_stamp: "Formatted at 2026-08-12 19:07:50 on dist-test-slave-49h3"
server_key: "e9f0cf7cbc40a30f8da875c25cc0209d"
server_key_iv: "67f9d2f1ad56493e10392d005991d37f"
server_key_version: "encryptionkey@0"
I20260812 19:07:50.077787 17145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance:
uuid: "80ad327fdca74735873497aacc657e5e"
format_stamp: "Formatted at 2026-08-12 19:07:50 on dist-test-slave-49h3"
server_key: "e9f0cf7cbc40a30f8da875c25cc0209d"
server_key_iv: "67f9d2f1ad56493e10392d005991d37f"
server_key_version: "encryptionkey@0"
I20260812 19:07:50.081496 17145 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 19:07:50.084065 17165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:50.085289 17145 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:07:50.085405 17145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "80ad327fdca74735873497aacc657e5e"
format_stamp: "Formatted at 2026-08-12 19:07:50 on dist-test-slave-49h3"
server_key: "e9f0cf7cbc40a30f8da875c25cc0209d"
server_key_iv: "67f9d2f1ad56493e10392d005991d37f"
server_key_version: "encryptionkey@0"
I20260812 19:07:50.085559 17145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:50.103287 17145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:50.106246 17145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:50.106432 17145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:50.107043 17145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:07:50.107887 17145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:07:50.107936 17145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:50.107991 17145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:07:50.108018 17145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:50.116721 17145 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:46489
I20260812 19:07:50.116750 17301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:46489 every 8 connection(s)
I20260812 19:07:50.117691 17145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:07:50.123867 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 17145
I20260812 19:07:50.123983 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance
I20260812 19:07:50.124258 26120 external_mini_cluster.cc:1428] Setting key c3dae556966a8925a7825fe876ea0ab7
Aug 12 19:07:50 dist-test-slave-49h3 krb5kdc[17015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561670, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.1@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:07:50.131639 17120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:50.119515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:57821 (local address 127.25.130.62:39425)
0812 19:07:50.119662 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:50.119665 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:50.120761 (+ 1096us) server_negotiation.cc:407] Connection header received
0812 19:07:50.121680 (+ 919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:50.121684 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:50.121746 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:50.121819 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:50.123016 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.123547 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.124888 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.125192 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.125278 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:50.127984 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:50.127987 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:50.127989 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:07:50.128014 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:50.129578 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:50.130097 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:50.130101 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:50.130102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:50.130183 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:50.130507 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:50.130510 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:50.130512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:50.130709 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:50.130797 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:50.131315 (+ 518us) server_negotiation.cc:299] Negotiation successful
0812 19:07:50.131465 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 19:07:50.132462 17304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:50.119881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:39425 (local address 127.25.130.1:57821)
0812 19:07:50.120595 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:50.120640 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:50.121439 (+ 799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:50.122001 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:50.122008 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:50.122369 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:07:50.122862 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:50.122877 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.123674 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.123700 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:50.124748 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:50.124760 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.125357 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.125361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:50.125600 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:50.126235 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:07:50.126253 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:07:50.127816 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:07:50.129701 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:50.129708 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:50.129725 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:50.129992 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:50.130267 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:50.130270 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:50.130272 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:50.130418 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:50.130811 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:07:50.130818 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:07:50.131080 (+ 262us) client_negotiation.cc:769] Sending connection context
0812 19:07:50.131253 (+ 173us) client_negotiation.cc:240] Negotiation successful
0812 19:07:50.131610 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":127,"threads_started":1}
I20260812 19:07:50.133888 17302 heartbeater.cc:347] Connected to a master server at 127.25.130.62:39425
I20260812 19:07:50.134128 17302 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.25.130.2@KRBTEST.COM; defaulting to no policy
I20260812 19:07:50.134636 17302 heartbeater.cc:511] Master 127.25.130.62:39425 requested a full tablet report, sending...
I20260812 19:07:50.136263 17067 ts_manager.cc:194] Registered new tserver with Master: 80ad327fdca74735873497aacc657e5e (127.25.130.1:46489)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.2@KRBTEST.COM" created.
I20260812 19:07:50.137578 17067 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.1@KRBTEST.COM'} at 127.25.130.1:57821
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
Entry for principal kudu/127.25.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab.
I20260812 19:07:50.150856 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:0
--local_ip_for_outbound_sockets=127.25.130.2
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:39425
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:44313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:07:50.265262 17307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:50.265558 17307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:50.265614 17307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:50.269806 17307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:50.269877 17307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:50.269946 17307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:07:50.275573 17307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:44313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.2.keytab
--principal=kudu/127.25.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:39425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:50.276708 17307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:50.277779 17307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:50.287504 17317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:50.287492 17315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:50.287497 17314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:50.287760 17307 server_base.cc:1034] running on GCE node
I20260812 19:07:50.288213 17307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:50.288897 17307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:50.290055 17307 hybrid_clock.cc:648] HybridClock initialized: now 1786561670290028 us; error 44 us; skew 500 ppm
Aug 12 19:07:50 dist-test-slave-49h3 krb5kdc[17015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561670, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:07:50.292675 17307 init.cc:377] Logged in from keytab as kudu/127.25.130.2@KRBTEST.COM (short username kudu)
I20260812 19:07:50.293646 17307 webserver.cc:460] Webserver started at http://127.25.130.2:41299/ using document root <none> and password file <none>
I20260812 19:07:50.294181 17307 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:50.294241 17307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:50.294420 17307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:50.296484 17307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-1/data/instance:
uuid: "9115451b7523497b9b3b5ea3c809a1cb"
format_stamp: "Formatted at 2026-08-12 19:07:50 on dist-test-slave-49h3"
server_key: "fd095932fb72a6d5e68c8bcb7b8de79a"
server_key_iv: "b5a101aadae68ea51f7db70df2cae487"
server_key_version: "encryptionkey@0"
I20260812 19:07:50.297024 17307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance:
uuid: "9115451b7523497b9b3b5ea3c809a1cb"
format_stamp: "Formatted at 2026-08-12 19:07:50 on dist-test-slave-49h3"
server_key: "fd095932fb72a6d5e68c8bcb7b8de79a"
server_key_iv: "b5a101aadae68ea51f7db70df2cae487"
server_key_version: "encryptionkey@0"
I20260812 19:07:50.300855 17307 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 19:07:50.303457 17325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:50.304816 17307 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 19:07:50.304958 17307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "9115451b7523497b9b3b5ea3c809a1cb"
format_stamp: "Formatted at 2026-08-12 19:07:50 on dist-test-slave-49h3"
server_key: "fd095932fb72a6d5e68c8bcb7b8de79a"
server_key_iv: "b5a101aadae68ea51f7db70df2cae487"
server_key_version: "encryptionkey@0"
I20260812 19:07:50.305071 17307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:50.315655 17307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:50.318545 17307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:50.318732 17307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:50.319315 17307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:07:50.320156 17307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:07:50.320205 17307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:50.320258 17307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:07:50.320284 17307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:50.328650 17307 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:42275
I20260812 19:07:50.328672 17454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:42275 every 8 connection(s)
I20260812 19:07:50.329661 17307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:07:50.338306 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 17307
I20260812 19:07:50.338449 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance
I20260812 19:07:50.338779 26120 external_mini_cluster.cc:1428] Setting key d7237318d1588cffcca6a1e151a7cdb0
Aug 12 19:07:50 dist-test-slave-49h3 krb5kdc[17015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561670, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.2@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:07:50.342998 17120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:50.331446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:34403 (local address 127.25.130.62:39425)
0812 19:07:50.331624 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:50.331629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:50.332682 (+ 1053us) server_negotiation.cc:407] Connection header received
0812 19:07:50.333560 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:50.333562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:50.333619 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:50.333695 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:50.334880 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.335413 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.336552 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.336773 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.336847 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:50.339408 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:50.339412 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:50.339414 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:07:50.339440 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:50.340848 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:50.341362 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:50.341365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:50.341367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:50.341479 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:50.341799 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:50.341802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:50.341804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:50.341994 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:50.342105 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:50.342676 (+ 571us) server_negotiation.cc:299] Negotiation successful
0812 19:07:50.342823 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 19:07:50.343791 17457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:50.331853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:39425 (local address 127.25.130.2:34403)
0812 19:07:50.332530 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:50.332573 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:50.333352 (+ 779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:50.333847 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:50.333854 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:50.334227 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:07:50.334745 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:50.334759 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.335533 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.335536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:50.336449 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:50.336459 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.336897 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.336899 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:50.337043 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:50.337648 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:07:50.337665 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:07:50.339240 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:07:50.340967 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:50.340974 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:50.340988 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:50.341260 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:50.341557 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:50.341561 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:50.341563 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:50.341707 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:50.342135 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:07:50.342143 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:07:50.342438 (+ 295us) client_negotiation.cc:769] Sending connection context
0812 19:07:50.342610 (+ 172us) client_negotiation.cc:240] Negotiation successful
0812 19:07:50.342888 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":116,"threads_started":1}
I20260812 19:07:50.344947 17455 heartbeater.cc:347] Connected to a master server at 127.25.130.62:39425
I20260812 19:07:50.345206 17455 heartbeater.cc:464] Registering TS with master...
I20260812 19:07:50.345816 17455 heartbeater.cc:511] Master 127.25.130.62:39425 requested a full tablet report, sending...
I20260812 19:07:50.346971 17067 ts_manager.cc:194] Registered new tserver with Master: 9115451b7523497b9b3b5ea3c809a1cb (127.25.130.2:42275)
I20260812 19:07:50.347770 17067 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.2@KRBTEST.COM'} at 127.25.130.2:34403
WARNING: no policy specified for kudu/127.25.130.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
Entry for principal kudu/127.25.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab.
I20260812 19:07:50.366569 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:0
--local_ip_for_outbound_sockets=127.25.130.3
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:39425
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:44313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:07:50.478075 17461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:50.478341 17461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:50.478389 17461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:50.482496 17461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:50.482573 17461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:50.482647 17461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:07:50.488242 17461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:44313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.3.keytab
--principal=kudu/127.25.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:39425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:50.489408 17461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:50.490509 17461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:50.500062 17469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:50.500063 17468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:50.500056 17471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:50.500277 17461 server_base.cc:1034] running on GCE node
I20260812 19:07:50.500808 17461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:50.501427 17461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:50.502619 17461 hybrid_clock.cc:648] HybridClock initialized: now 1786561670502600 us; error 32 us; skew 500 ppm
Aug 12 19:07:50 dist-test-slave-49h3 krb5kdc[17015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561670, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:07:50.505322 17461 init.cc:377] Logged in from keytab as kudu/127.25.130.3@KRBTEST.COM (short username kudu)
I20260812 19:07:50.506345 17461 webserver.cc:460] Webserver started at http://127.25.130.3:37393/ using document root <none> and password file <none>
I20260812 19:07:50.506915 17461 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:50.506985 17461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:50.507211 17461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:50.509418 17461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-2/data/instance:
uuid: "7adb6a7669a84cacbd758151b0b5abf8"
format_stamp: "Formatted at 2026-08-12 19:07:50 on dist-test-slave-49h3"
server_key: "49f15328821fca434b06a32e85d1f2ad"
server_key_iv: "e7df34809699b1080538f23bb6da400b"
server_key_version: "encryptionkey@0"
I20260812 19:07:50.510021 17461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance:
uuid: "7adb6a7669a84cacbd758151b0b5abf8"
format_stamp: "Formatted at 2026-08-12 19:07:50 on dist-test-slave-49h3"
server_key: "49f15328821fca434b06a32e85d1f2ad"
server_key_iv: "e7df34809699b1080538f23bb6da400b"
server_key_version: "encryptionkey@0"
I20260812 19:07:50.513947 17461 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 19:07:50.516472 17481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:50.517753 17461 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:07:50.517865 17461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "7adb6a7669a84cacbd758151b0b5abf8"
format_stamp: "Formatted at 2026-08-12 19:07:50 on dist-test-slave-49h3"
server_key: "49f15328821fca434b06a32e85d1f2ad"
server_key_iv: "e7df34809699b1080538f23bb6da400b"
server_key_version: "encryptionkey@0"
I20260812 19:07:50.517966 17461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:50.530758 17461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:50.533823 17461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:50.534020 17461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:50.534631 17461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:07:50.535478 17461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:07:50.535529 17461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:50.535584 17461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:07:50.535611 17461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:50.544106 17461 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:41637
I20260812 19:07:50.544137 17627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:41637 every 8 connection(s)
I20260812 19:07:50.544984 17461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:07:50.553689 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 17461
I20260812 19:07:50.553814 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance
Aug 12 19:07:50 dist-test-slave-49h3 krb5kdc[17015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561670, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.3@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:07:50.554075 26120 external_mini_cluster.cc:1428] Setting key 63db7902a835e069612c8904affbd887
I20260812 19:07:50.557981 17120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:50.546818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:39479 (local address 127.25.130.62:39425)
0812 19:07:50.546966 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:50.546970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:50.548043 (+ 1073us) server_negotiation.cc:407] Connection header received
0812 19:07:50.548857 (+ 814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:50.548860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:50.548907 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:50.548980 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:50.550201 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.550732 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.551850 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.552085 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.552168 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:50.554623 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:50.554627 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:50.554629 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:07:50.554659 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:50.556088 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:50.556592 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:50.556595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:50.556596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:50.556684 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:50.556949 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:50.556951 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:50.556952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:50.557124 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:50.557198 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:50.557762 (+ 564us) server_negotiation.cc:299] Negotiation successful
0812 19:07:50.557862 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 19:07:50.558885 17632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:50.547210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:39425 (local address 127.25.130.3:39479)
0812 19:07:50.547894 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:50.547936 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:50.548690 (+ 754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:50.549141 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:50.549147 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:50.549553 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:07:50.550040 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:50.550053 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.550844 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.550846 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:50.551731 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:50.551741 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.552227 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.552230 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:50.552374 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:50.552901 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:07:50.552917 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:07:50.554479 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:07:50.556199 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:50.556205 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:50.556220 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:50.556478 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:50.556760 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:07:50.556762 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:07:50.556763 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:07:50.556885 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:07:50.557233 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:07:50.557238 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:07:50.557535 (+ 297us) client_negotiation.cc:769] Sending connection context
0812 19:07:50.557702 (+ 167us) client_negotiation.cc:240] Negotiation successful
0812 19:07:50.557960 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":115,"threads_started":1}
I20260812 19:07:50.560042 17628 heartbeater.cc:347] Connected to a master server at 127.25.130.62:39425
I20260812 19:07:50.560298 17628 heartbeater.cc:464] Registering TS with master...
I20260812 19:07:50.560905 17628 heartbeater.cc:511] Master 127.25.130.62:39425 requested a full tablet report, sending...
I20260812 19:07:50.562083 17067 ts_manager.cc:194] Registered new tserver with Master: 7adb6a7669a84cacbd758151b0b5abf8 (127.25.130.3:41637)
I20260812 19:07:50.562832 17067 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.130.3@KRBTEST.COM'} at 127.25.130.3:39479
I20260812 19:07:50.567795 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:07:50.576386 17120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:50.569757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54312 (local address 127.25.130.62:39425)
0812 19:07:50.569861 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:50.569864 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:50.570029 (+ 165us) server_negotiation.cc:407] Connection header received
0812 19:07:50.570140 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:50.570142 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:50.570182 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:50.570244 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:50.570881 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.571393 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.572137 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.572333 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.572398 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:50.573619 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:50.573622 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:50.573624 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:07:50.573649 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:50.574789 (+ 1140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:50.575225 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:50.575228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:50.575230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:50.575318 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:50.575553 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:50.575556 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:50.575557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:50.575779 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:50.575881 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:50.576161 (+ 280us) server_negotiation.cc:299] Negotiation successful
0812 19:07:50.576264 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 19:07:50.579073 17067 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54312:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:07:50.582194 17067 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:07:50.596711 17648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:50.590619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:41637 (local address 127.0.0.1:53428)
0812 19:07:50.591805 (+ 1186us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:50.591817 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:50.591900 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:50.592527 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:50.592532 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:50.592553 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:50.592708 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:50.592716 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.593832 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.593836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:50.595324 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:50.595336 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.596161 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.596164 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:50.596277 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:50.596318 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 19:07:50.596419 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 19:07:50.596542 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1128,"mutex_wait_us":35,"thread_start_us":90,"threads_started":1}
I20260812 19:07:50.596923 17649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:50.591373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35152 (local address 127.25.130.1:46489)
0812 19:07:50.591881 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:50.591895 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:50.592140 (+ 245us) server_negotiation.cc:407] Connection header received
0812 19:07:50.592210 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:50.592214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:50.592353 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:50.592515 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:50.592978 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.593833 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.595452 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.596275 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.596362 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:50.596496 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:50.596650 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 19:07:50.596747 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":123,"threads_started":1}
I20260812 19:07:50.597396 17645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:50.590870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:46489 (local address 127.0.0.1:35152)
0812 19:07:50.592041 (+ 1171us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:50.592050 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:50.592124 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:50.592699 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:50.592702 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:50.592731 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:50.592860 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:50.592867 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.594003 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.594007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:50.595324 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:50.595336 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.596386 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.596390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:50.596483 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:50.596504 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 19:07:50.597146 (+ 642us) client_negotiation.cc:240] Negotiation successful
0812 19:07:50.597224 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1127,"mutex_wait_us":32,"spinlock_wait_cycles":6400,"thread_start_us":82,"threads_started":1}
I20260812 19:07:50.597424 17650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:50.591384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53428 (local address 127.25.130.3:41637)
0812 19:07:50.592038 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:50.592045 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:50.592065 (+ 20us) server_negotiation.cc:407] Connection header received
0812 19:07:50.592139 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:50.592144 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:50.592352 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:50.592525 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:50.592805 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.593703 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.595482 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.596028 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.596138 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:50.596292 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:50.596603 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 19:07:50.596706 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":163,"threads_started":1}
I20260812 19:07:50.597733 17644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:50.590553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:42275 (local address 127.0.0.1:52742)
0812 19:07:50.591415 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:50.591471 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:50.591618 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:50.592699 (+ 1081us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:50.592702 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:50.592733 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:50.593899 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:50.593907 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.595093 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.595096 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:50.596113 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:50.596125 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.597149 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.597153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:50.597253 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:50.597274 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 19:07:50.597335 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 19:07:50.597395 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"mutex_wait_us":1064,"spinlock_wait_cycles":3200,"thread_start_us":102,"threads_started":1}
I20260812 19:07:50.598495 17647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:50.590715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52742 (local address 127.25.130.2:42275)
0812 19:07:50.592314 (+ 1599us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:50.592322 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:50.592341 (+ 19us) server_negotiation.cc:407] Connection header received
0812 19:07:50.592399 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:50.592403 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:50.592580 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:50.592722 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:50.594034 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.594966 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.596263 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.597043 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.598065 (+ 1022us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:50.598213 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:50.598265 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 19:07:50.598355 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1513,"thread_start_us":166,"threads_started":1}
I20260812 19:07:50.600347 17222 tablet_service.cc:1467] Processing CreateTablet for tablet 6e8003200da34e04beebba6b8c3e9112 (DEFAULT_TABLE table=test-table [id=79986535728e4974afd4ba2681b44fca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:07:50.600463 17543 tablet_service.cc:1467] Processing CreateTablet for tablet 6e8003200da34e04beebba6b8c3e9112 (DEFAULT_TABLE table=test-table [id=79986535728e4974afd4ba2681b44fca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:07:50.601423 17222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e8003200da34e04beebba6b8c3e9112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:50.601117 17379 tablet_service.cc:1467] Processing CreateTablet for tablet 6e8003200da34e04beebba6b8c3e9112 (DEFAULT_TABLE table=test-table [id=79986535728e4974afd4ba2681b44fca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:07:50.601567 17543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e8003200da34e04beebba6b8c3e9112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:50.602115 17379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e8003200da34e04beebba6b8c3e9112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:50.607896 17651 tablet_bootstrap.cc:492] T 6e8003200da34e04beebba6b8c3e9112 P 80ad327fdca74735873497aacc657e5e: Bootstrap starting.
I20260812 19:07:50.609726 17653 tablet_bootstrap.cc:492] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb: Bootstrap starting.
I20260812 19:07:50.610411 17651 tablet_bootstrap.cc:654] T 6e8003200da34e04beebba6b8c3e9112 P 80ad327fdca74735873497aacc657e5e: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:50.611543 17651 log.cc:824] T 6e8003200da34e04beebba6b8c3e9112 P 80ad327fdca74735873497aacc657e5e: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:50.611945 17653 tablet_bootstrap.cc:654] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:50.612337 17652 tablet_bootstrap.cc:492] T 6e8003200da34e04beebba6b8c3e9112 P 7adb6a7669a84cacbd758151b0b5abf8: Bootstrap starting.
I20260812 19:07:50.612833 17653 log.cc:824] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:50.613956 17651 tablet_bootstrap.cc:492] T 6e8003200da34e04beebba6b8c3e9112 P 80ad327fdca74735873497aacc657e5e: No bootstrap required, opened a new log
I20260812 19:07:50.614210 17651 ts_tablet_manager.cc:1397] T 6e8003200da34e04beebba6b8c3e9112 P 80ad327fdca74735873497aacc657e5e: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260812 19:07:50.614517 17653 tablet_bootstrap.cc:492] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb: No bootstrap required, opened a new log
I20260812 19:07:50.614552 17652 tablet_bootstrap.cc:654] T 6e8003200da34e04beebba6b8c3e9112 P 7adb6a7669a84cacbd758151b0b5abf8: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:50.614693 17653 ts_tablet_manager.cc:1397] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 19:07:50.615722 17652 log.cc:824] T 6e8003200da34e04beebba6b8c3e9112 P 7adb6a7669a84cacbd758151b0b5abf8: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:50.617863 17652 tablet_bootstrap.cc:492] T 6e8003200da34e04beebba6b8c3e9112 P 7adb6a7669a84cacbd758151b0b5abf8: No bootstrap required, opened a new log
I20260812 19:07:50.617749 17653 raft_consensus.cc:348] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9115451b7523497b9b3b5ea3c809a1cb" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 42275 } } peers { permanent_uuid: "7adb6a7669a84cacbd758151b0b5abf8" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 41637 } } peers { permanent_uuid: "80ad327fdca74735873497aacc657e5e" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 46489 } }
I20260812 19:07:50.617997 17653 raft_consensus.cc:374] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:50.618049 17653 raft_consensus.cc:729] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9115451b7523497b9b3b5ea3c809a1cb, State: Initialized, Role: FOLLOWER
I20260812 19:07:50.618064 17652 ts_tablet_manager.cc:1397] T 6e8003200da34e04beebba6b8c3e9112 P 7adb6a7669a84cacbd758151b0b5abf8: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 19:07:50.618050 17651 raft_consensus.cc:348] T 6e8003200da34e04beebba6b8c3e9112 P 80ad327fdca74735873497aacc657e5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9115451b7523497b9b3b5ea3c809a1cb" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 42275 } } peers { permanent_uuid: "7adb6a7669a84cacbd758151b0b5abf8" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 41637 } } peers { permanent_uuid: "80ad327fdca74735873497aacc657e5e" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 46489 } }
I20260812 19:07:50.618305 17651 raft_consensus.cc:374] T 6e8003200da34e04beebba6b8c3e9112 P 80ad327fdca74735873497aacc657e5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:50.618369 17651 raft_consensus.cc:729] T 6e8003200da34e04beebba6b8c3e9112 P 80ad327fdca74735873497aacc657e5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80ad327fdca74735873497aacc657e5e, State: Initialized, Role: FOLLOWER
I20260812 19:07:50.618507 17653 consensus_queue.cc:260] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9115451b7523497b9b3b5ea3c809a1cb" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 42275 } } peers { permanent_uuid: "7adb6a7669a84cacbd758151b0b5abf8" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 41637 } } peers { permanent_uuid: "80ad327fdca74735873497aacc657e5e" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 46489 } }
I20260812 19:07:50.618958 17651 consensus_queue.cc:260] T 6e8003200da34e04beebba6b8c3e9112 P 80ad327fdca74735873497aacc657e5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9115451b7523497b9b3b5ea3c809a1cb" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 42275 } } peers { permanent_uuid: "7adb6a7669a84cacbd758151b0b5abf8" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 41637 } } peers { permanent_uuid: "80ad327fdca74735873497aacc657e5e" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 46489 } }
I20260812 19:07:50.619177 17455 heartbeater.cc:503] Master 127.25.130.62:39425 was elected leader, sending a full tablet report...
I20260812 19:07:50.619560 17653 ts_tablet_manager.cc:1428] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 19:07:50.619902 17651 ts_tablet_manager.cc:1428] T 6e8003200da34e04beebba6b8c3e9112 P 80ad327fdca74735873497aacc657e5e: Time spent starting tablet: real 0.006s user 0.001s sys 0.004s
I20260812 19:07:50.619982 17302 heartbeater.cc:503] Master 127.25.130.62:39425 was elected leader, sending a full tablet report...
I20260812 19:07:50.621691 17652 raft_consensus.cc:348] T 6e8003200da34e04beebba6b8c3e9112 P 7adb6a7669a84cacbd758151b0b5abf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9115451b7523497b9b3b5ea3c809a1cb" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 42275 } } peers { permanent_uuid: "7adb6a7669a84cacbd758151b0b5abf8" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 41637 } } peers { permanent_uuid: "80ad327fdca74735873497aacc657e5e" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 46489 } }
I20260812 19:07:50.621912 17652 raft_consensus.cc:374] T 6e8003200da34e04beebba6b8c3e9112 P 7adb6a7669a84cacbd758151b0b5abf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:50.621965 17652 raft_consensus.cc:729] T 6e8003200da34e04beebba6b8c3e9112 P 7adb6a7669a84cacbd758151b0b5abf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7adb6a7669a84cacbd758151b0b5abf8, State: Initialized, Role: FOLLOWER
I20260812 19:07:50.622364 17652 consensus_queue.cc:260] T 6e8003200da34e04beebba6b8c3e9112 P 7adb6a7669a84cacbd758151b0b5abf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9115451b7523497b9b3b5ea3c809a1cb" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 42275 } } peers { permanent_uuid: "7adb6a7669a84cacbd758151b0b5abf8" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 41637 } } peers { permanent_uuid: "80ad327fdca74735873497aacc657e5e" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 46489 } }
I20260812 19:07:50.623107 17628 heartbeater.cc:503] Master 127.25.130.62:39425 was elected leader, sending a full tablet report...
I20260812 19:07:50.623366 17652 ts_tablet_manager.cc:1428] T 6e8003200da34e04beebba6b8c3e9112 P 7adb6a7669a84cacbd758151b0b5abf8: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 19:07:50.760196 17658 raft_consensus.cc:482] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:50.760447 17658 raft_consensus.cc:504] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9115451b7523497b9b3b5ea3c809a1cb" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 42275 } } peers { permanent_uuid: "7adb6a7669a84cacbd758151b0b5abf8" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 41637 } } peers { permanent_uuid: "80ad327fdca74735873497aacc657e5e" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 46489 } }
I20260812 19:07:50.761700 17658 leader_election.cc:288] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7adb6a7669a84cacbd758151b0b5abf8 (127.25.130.3:41637), 80ad327fdca74735873497aacc657e5e (127.25.130.1:46489)
I20260812 19:07:50.766490 17649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:50.762225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:60349 (local address 127.25.130.1:46489)
0812 19:07:50.762352 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:50.762356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:50.762495 (+ 139us) server_negotiation.cc:407] Connection header received
0812 19:07:50.762617 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:50.762620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:50.762676 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:50.762742 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:50.762980 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.763689 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.765139 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.765783 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.765915 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:50.765969 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:50.766274 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 19:07:50.766354 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 19:07:50.766490 17650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:50.762053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:59373 (local address 127.25.130.3:41637)
0812 19:07:50.762183 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:50.762187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:50.762204 (+ 17us) server_negotiation.cc:407] Connection header received
0812 19:07:50.762376 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:50.762380 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:50.762428 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:50.762499 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:50.762784 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.763557 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.765142 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.765783 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.765915 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:50.765969 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:50.766274 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 19:07:50.766354 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 19:07:50.766498 17457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:50.761955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:41637 (local address 127.25.130.2:59373)
0812 19:07:50.762119 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:50.762135 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:50.762239 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:50.762529 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:50.762532 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:50.762548 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:50.762675 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:50.762680 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.763668 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.763670 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:50.765035 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:50.765042 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.765941 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.765945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:50.766158 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:50.766175 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 19:07:50.766239 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 19:07:50.766328 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"mutex_wait_us":83}
I20260812 19:07:50.766507 17661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:50.761981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:46489 (local address 127.25.130.2:60349)
0812 19:07:50.762408 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:50.762418 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:50.762518 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:50.762772 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:50.762774 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:50.762791 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:50.762885 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:50.762890 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.763782 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.763785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:50.765016 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:50.765030 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.765941 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.765945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:50.766093 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:50.766141 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 19:07:50.766228 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 19:07:50.766328 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":68,"thread_start_us":142,"threads_started":1}
I20260812 19:07:50.767164 17572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e8003200da34e04beebba6b8c3e9112" candidate_uuid: "9115451b7523497b9b3b5ea3c809a1cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7adb6a7669a84cacbd758151b0b5abf8" is_pre_election: true
I20260812 19:07:50.767177 17247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e8003200da34e04beebba6b8c3e9112" candidate_uuid: "9115451b7523497b9b3b5ea3c809a1cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80ad327fdca74735873497aacc657e5e" is_pre_election: true
I20260812 19:07:50.767462 17572 raft_consensus.cc:2454] T 6e8003200da34e04beebba6b8c3e9112 P 7adb6a7669a84cacbd758151b0b5abf8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9115451b7523497b9b3b5ea3c809a1cb in term 0.
I20260812 19:07:50.767462 17247 raft_consensus.cc:2454] T 6e8003200da34e04beebba6b8c3e9112 P 80ad327fdca74735873497aacc657e5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9115451b7523497b9b3b5ea3c809a1cb in term 0.
I20260812 19:07:50.767968 17330 leader_election.cc:302] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7adb6a7669a84cacbd758151b0b5abf8, 80ad327fdca74735873497aacc657e5e, 9115451b7523497b9b3b5ea3c809a1cb; no voters:
I20260812 19:07:50.768254 17658 raft_consensus.cc:2781] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:07:50.768349 17658 raft_consensus.cc:482] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:50.768397 17658 raft_consensus.cc:3037] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:50.769488 17658 raft_consensus.cc:504] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9115451b7523497b9b3b5ea3c809a1cb" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 42275 } } peers { permanent_uuid: "7adb6a7669a84cacbd758151b0b5abf8" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 41637 } } peers { permanent_uuid: "80ad327fdca74735873497aacc657e5e" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 46489 } }
I20260812 19:07:50.769841 17658 leader_election.cc:288] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb [CANDIDATE]: Term 1 election: Requested vote from peers 7adb6a7669a84cacbd758151b0b5abf8 (127.25.130.3:41637), 80ad327fdca74735873497aacc657e5e (127.25.130.1:46489)
I20260812 19:07:50.770228 17572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e8003200da34e04beebba6b8c3e9112" candidate_uuid: "9115451b7523497b9b3b5ea3c809a1cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7adb6a7669a84cacbd758151b0b5abf8"
I20260812 19:07:50.770228 17247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e8003200da34e04beebba6b8c3e9112" candidate_uuid: "9115451b7523497b9b3b5ea3c809a1cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80ad327fdca74735873497aacc657e5e"
I20260812 19:07:50.770361 17572 raft_consensus.cc:3037] T 6e8003200da34e04beebba6b8c3e9112 P 7adb6a7669a84cacbd758151b0b5abf8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:50.770361 17247 raft_consensus.cc:3037] T 6e8003200da34e04beebba6b8c3e9112 P 80ad327fdca74735873497aacc657e5e [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:50.771627 17572 raft_consensus.cc:2454] T 6e8003200da34e04beebba6b8c3e9112 P 7adb6a7669a84cacbd758151b0b5abf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9115451b7523497b9b3b5ea3c809a1cb in term 1.
I20260812 19:07:50.771627 17247 raft_consensus.cc:2454] T 6e8003200da34e04beebba6b8c3e9112 P 80ad327fdca74735873497aacc657e5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9115451b7523497b9b3b5ea3c809a1cb in term 1.
I20260812 19:07:50.771950 17330 leader_election.cc:302] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7adb6a7669a84cacbd758151b0b5abf8, 9115451b7523497b9b3b5ea3c809a1cb; no voters:
I20260812 19:07:50.772137 17658 raft_consensus.cc:2781] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:07:50.772375 17658 raft_consensus.cc:686] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb [term 1 LEADER]: Becoming Leader. State: Replica: 9115451b7523497b9b3b5ea3c809a1cb, State: Running, Role: LEADER
I20260812 19:07:50.772701 17658 consensus_queue.cc:237] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9115451b7523497b9b3b5ea3c809a1cb" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 42275 } } peers { permanent_uuid: "7adb6a7669a84cacbd758151b0b5abf8" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 41637 } } peers { permanent_uuid: "80ad327fdca74735873497aacc657e5e" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 46489 } }
I20260812 19:07:50.775904 17065 catalog_manager.cc:5495] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 9115451b7523497b9b3b5ea3c809a1cb (127.25.130.2). New cstate: current_term: 1 leader_uuid: "9115451b7523497b9b3b5ea3c809a1cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9115451b7523497b9b3b5ea3c809a1cb" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 42275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7adb6a7669a84cacbd758151b0b5abf8" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 41637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80ad327fdca74735873497aacc657e5e" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 46489 } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:50.786590 17647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:50.783406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52758 (local address 127.25.130.2:42275)
0812 19:07:50.783529 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:50.783533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:50.783545 (+ 12us) server_negotiation.cc:407] Connection header received
0812 19:07:50.783616 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:50.783618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:50.783673 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:50.783758 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:07:50.783942 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.784431 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.785206 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:50.785419 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:50.785553 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:50.785784 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:07:50.786235 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:07:50.786326 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:50.786391 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 19:07:50.786445 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 19:07:50.792716 17247 raft_consensus.cc:1261] T 6e8003200da34e04beebba6b8c3e9112 P 80ad327fdca74735873497aacc657e5e [term 1 FOLLOWER]: Refusing update from remote peer 9115451b7523497b9b3b5ea3c809a1cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:50.792716 17572 raft_consensus.cc:1261] T 6e8003200da34e04beebba6b8c3e9112 P 7adb6a7669a84cacbd758151b0b5abf8 [term 1 FOLLOWER]: Refusing update from remote peer 9115451b7523497b9b3b5ea3c809a1cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:50.793609 17662 consensus_queue.cc:1035] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "80ad327fdca74735873497aacc657e5e" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 46489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:07:50.793821 17658 consensus_queue.cc:1035] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7adb6a7669a84cacbd758151b0b5abf8" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 41637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 19:07:50.796002 17631 tablet.cc:2369] T 6e8003200da34e04beebba6b8c3e9112 P 7adb6a7669a84cacbd758151b0b5abf8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:07:50.801151 17667 mvcc.cc:205] Tried to move back new op lower bound from 7317756603559014400 to 7317756603487543296. Current Snapshot: MvccSnapshot[applied={T|T < 7317756603559014400}]
I20260812 19:07:50.804592 17670 mvcc.cc:205] Tried to move back new op lower bound from 7317756603559014400 to 7317756603487543296. Current Snapshot: MvccSnapshot[applied={T|T < 7317756603559014400}]
I20260812 19:07:50.810799 17065 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:07:50.810979 17065 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:07:50.813555 17065 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 88ee3383b9234d0eaf68e77e781f54ed: Sending DeleteTablet for 3 replicas of tablet 6e8003200da34e04beebba6b8c3e9112
I20260812 19:07:50.814405 17379 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e8003200da34e04beebba6b8c3e9112 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:07:50 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:52742
I20260812 19:07:50.814409 17543 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e8003200da34e04beebba6b8c3e9112 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:07:50 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:53428
I20260812 19:07:50.814996 17676 tablet_replica.cc:330] T 6e8003200da34e04beebba6b8c3e9112 P 7adb6a7669a84cacbd758151b0b5abf8: stopping tablet replica
I20260812 19:07:50.814980 17222 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e8003200da34e04beebba6b8c3e9112 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:07:50 UTC) from {username='kudu', principal='kudu/127.25.130.62@KRBTEST.COM'} at 127.0.0.1:35152
I20260812 19:07:50.815362 17676 raft_consensus.cc:2229] T 6e8003200da34e04beebba6b8c3e9112 P 7adb6a7669a84cacbd758151b0b5abf8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:50.815425 17679 tablet_replica.cc:330] T 6e8003200da34e04beebba6b8c3e9112 P 80ad327fdca74735873497aacc657e5e: stopping tablet replica
I20260812 19:07:50.815702 17676 raft_consensus.cc:2258] T 6e8003200da34e04beebba6b8c3e9112 P 7adb6a7669a84cacbd758151b0b5abf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:50.815726 17679 raft_consensus.cc:2229] T 6e8003200da34e04beebba6b8c3e9112 P 80ad327fdca74735873497aacc657e5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:50.816013 17679 raft_consensus.cc:2258] T 6e8003200da34e04beebba6b8c3e9112 P 80ad327fdca74735873497aacc657e5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:50.816504 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 17145
I20260812 19:07:50.817345 17676 ts_tablet_manager.cc:1905] T 6e8003200da34e04beebba6b8c3e9112 P 7adb6a7669a84cacbd758151b0b5abf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:07:50.820094 17676 ts_tablet_manager.cc:1918] T 6e8003200da34e04beebba6b8c3e9112 P 7adb6a7669a84cacbd758151b0b5abf8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:07:50.820205 17676 log.cc:1196] T 6e8003200da34e04beebba6b8c3e9112 P 7adb6a7669a84cacbd758151b0b5abf8: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-2/wal/wals/6e8003200da34e04beebba6b8c3e9112
I20260812 19:07:50.820538 17676 ts_tablet_manager.cc:1939] T 6e8003200da34e04beebba6b8c3e9112 P 7adb6a7669a84cacbd758151b0b5abf8: Deleting consensus metadata
I20260812 19:07:50.821931 17677 tablet_replica.cc:330] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb: stopping tablet replica
I20260812 19:07:50.821992 17051 catalog_manager.cc:4841] TS 7adb6a7669a84cacbd758151b0b5abf8 (127.25.130.3:41637): tablet 6e8003200da34e04beebba6b8c3e9112 (table test-table [id=79986535728e4974afd4ba2681b44fca]) successfully deleted
I20260812 19:07:50.822255 17677 raft_consensus.cc:2229] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:07:50.822649 17677 raft_consensus.cc:2258] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:50.824222 17677 ts_tablet_manager.cc:1905] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:07:50.826958 17677 ts_tablet_manager.cc:1918] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:07:50.827064 17677 log.cc:1196] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561415988404-26120-0/minicluster-data/ts-1/wal/wals/6e8003200da34e04beebba6b8c3e9112
I20260812 19:07:50.827387 17677 ts_tablet_manager.cc:1939] T 6e8003200da34e04beebba6b8c3e9112 P 9115451b7523497b9b3b5ea3c809a1cb: Deleting consensus metadata
I20260812 19:07:50.828627 17048 catalog_manager.cc:4841] TS 9115451b7523497b9b3b5ea3c809a1cb (127.25.130.2:42275): tablet 6e8003200da34e04beebba6b8c3e9112 (table test-table [id=79986535728e4974afd4ba2681b44fca]) successfully deleted
W20260812 19:07:50.828685 17049 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.130.1:46489 (error 108)
I20260812 19:07:50.829494 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 17307
I20260812 19:07:50.829730 17645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:50.829441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:46489 (local address 127.0.0.1:35156)
0812 19:07:50.829574 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:50.829646 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.1:46489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 19:07:50.833802 17049 catalog_manager.cc:4584] TS 80ad327fdca74735873497aacc657e5e (127.25.130.1:46489): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6e8003200da34e04beebba6b8c3e9112: Network error: Client connection negotiation failed: client connection to 127.25.130.1:46489: connect: Connection refused (error 111)
I20260812 19:07:50.839919 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 17461
I20260812 19:07:50.849548 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 17030
2026-08-12T19:07:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2375 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T19:07:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:07:50Z Disabled control of system clock
I20260812 19:07:50.882733 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:33101
--webserver_interface=127.25.130.62
--webserver_port=0
--builtin_ntp_servers=127.25.130.20:45029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:33101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 19:07:50.995918 17684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:50.996218 17684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:50.996256 17684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:51.000458 17684 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:07:51.000535 17684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:51.000550 17684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:51.000566 17684 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:07:51.000581 17684 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:07:51.005609 17684 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:45029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:33101
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:33101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:51.006785 17684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:51.008234 17684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:51.016943 17694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:51.016973 17692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:51.016943 17691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:51.017014 17684 server_base.cc:1034] running on GCE node
I20260812 19:07:51.017896 17684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:51.019091 17684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:51.020264 17684 hybrid_clock.cc:648] HybridClock initialized: now 1786561671020248 us; error 34 us; skew 500 ppm
I20260812 19:07:51.022332 17684 webserver.cc:460] Webserver started at http://127.25.130.62:42919/ using document root <none> and password file <none>
I20260812 19:07:51.022951 17684 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:51.023033 17684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:51.023254 17684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:51.025542 17684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "7c5af1c52ad0463991e503aa9435df70"
format_stamp: "Formatted at 2026-08-12 19:07:51 on dist-test-slave-49h3"
server_key: "52c3e1918193d89550f3718cd0c239aa"
server_key_iv: "4ac12551d6aa1bf886f1bbe4a1e56c1b"
server_key_version: "encryptionkey@0"
I20260812 19:07:51.026191 17684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "7c5af1c52ad0463991e503aa9435df70"
format_stamp: "Formatted at 2026-08-12 19:07:51 on dist-test-slave-49h3"
server_key: "52c3e1918193d89550f3718cd0c239aa"
server_key_iv: "4ac12551d6aa1bf886f1bbe4a1e56c1b"
server_key_version: "encryptionkey@0"
I20260812 19:07:51.030337 17684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 19:07:51.033051 17701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:51.034474 17684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:07:51.034626 17684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "7c5af1c52ad0463991e503aa9435df70"
format_stamp: "Formatted at 2026-08-12 19:07:51 on dist-test-slave-49h3"
server_key: "52c3e1918193d89550f3718cd0c239aa"
server_key_iv: "4ac12551d6aa1bf886f1bbe4a1e56c1b"
server_key_version: "encryptionkey@0"
I20260812 19:07:51.034749 17684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:51.059576 17684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:51.060372 17684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:51.060563 17684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:51.067512 17684 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:33101
I20260812 19:07:51.067523 17771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:33101 every 8 connection(s)
I20260812 19:07:51.068481 17684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:07:51.069258 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 17684
I20260812 19:07:51.069399 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:07:51.069742 26120 external_mini_cluster.cc:1428] Setting key 78e9cbbbabb9f2bf7ad95ba6fae81380
I20260812 19:07:51.072170 17772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:51.077984 17772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c5af1c52ad0463991e503aa9435df70: Bootstrap starting.
I20260812 19:07:51.079113 17775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:51.071188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47456 (local address 127.25.130.62:33101)
0812 19:07:51.071856 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:51.071872 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:51.071908 (+ 36us) server_negotiation.cc:407] Connection header received
0812 19:07:51.072577 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:51.072604 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:51.073027 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:51.073342 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:51.073802 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.074801 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.075660 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.076137 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.076282 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:51.077285 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:51.077291 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:51.077305 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:07:51.077747 (+ 442us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:51.077807 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:07:51.077829 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:51.077914 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:51.078056 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 19:07:51.078355 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":153,"threads_started":1}
I20260812 19:07:51.080292 17772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c5af1c52ad0463991e503aa9435df70: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:51.081067 17772 log.cc:824] T 00000000000000000000000000000000 P 7c5af1c52ad0463991e503aa9435df70: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:51.083019 17772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c5af1c52ad0463991e503aa9435df70: No bootstrap required, opened a new log
I20260812 19:07:51.086541 17772 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c5af1c52ad0463991e503aa9435df70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c5af1c52ad0463991e503aa9435df70" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 33101 } }
I20260812 19:07:51.086783 17772 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c5af1c52ad0463991e503aa9435df70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:51.086843 17772 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c5af1c52ad0463991e503aa9435df70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c5af1c52ad0463991e503aa9435df70, State: Initialized, Role: FOLLOWER
I20260812 19:07:51.087253 17772 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c5af1c52ad0463991e503aa9435df70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c5af1c52ad0463991e503aa9435df70" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 33101 } }
I20260812 19:07:51.087385 17772 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c5af1c52ad0463991e503aa9435df70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:07:51.087450 17772 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c5af1c52ad0463991e503aa9435df70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:07:51.087549 17772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c5af1c52ad0463991e503aa9435df70 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:51.088614 17772 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c5af1c52ad0463991e503aa9435df70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c5af1c52ad0463991e503aa9435df70" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 33101 } }
I20260812 19:07:51.088948 17772 leader_election.cc:302] T 00000000000000000000000000000000 P 7c5af1c52ad0463991e503aa9435df70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c5af1c52ad0463991e503aa9435df70; no voters:
I20260812 19:07:51.089257 17772 leader_election.cc:288] T 00000000000000000000000000000000 P 7c5af1c52ad0463991e503aa9435df70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:07:51.089378 17777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c5af1c52ad0463991e503aa9435df70 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:07:51.089617 17777 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c5af1c52ad0463991e503aa9435df70 [term 1 LEADER]: Becoming Leader. State: Replica: 7c5af1c52ad0463991e503aa9435df70, State: Running, Role: LEADER
I20260812 19:07:51.089926 17777 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c5af1c52ad0463991e503aa9435df70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c5af1c52ad0463991e503aa9435df70" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 33101 } }
I20260812 19:07:51.090488 17772 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c5af1c52ad0463991e503aa9435df70 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:07:51.091041 17777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c5af1c52ad0463991e503aa9435df70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c5af1c52ad0463991e503aa9435df70. Latest consensus state: current_term: 1 leader_uuid: "7c5af1c52ad0463991e503aa9435df70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c5af1c52ad0463991e503aa9435df70" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 33101 } } }
I20260812 19:07:51.091168 17777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c5af1c52ad0463991e503aa9435df70 [sys.catalog]: This master's current role is: LEADER
I20260812 19:07:51.091531 17779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c5af1c52ad0463991e503aa9435df70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c5af1c52ad0463991e503aa9435df70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c5af1c52ad0463991e503aa9435df70" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 33101 } } }
I20260812 19:07:51.091599 17783 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:07:51.091627 17779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c5af1c52ad0463991e503aa9435df70 [sys.catalog]: This master's current role is: LEADER
I20260812 19:07:51.094661 17783 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:07:51.100675 17783 catalog_manager.cc:1347] Generated new cluster ID: 817206b4586f49158c492737b645981d
I20260812 19:07:51.100764 17783 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:07:51.123155 17783 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:07:51.124156 17783 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:07:51.131788 17783 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c5af1c52ad0463991e503aa9435df70: Generated new TSK 0
I20260812 19:07:51.132552 17783 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:07:51.140683 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.1:0
--local_ip_for_outbound_sockets=127.25.130.1
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:33101
--builtin_ntp_servers=127.25.130.20:45029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 19:07:51.276208 17799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:51.276513 17799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:51.276553 17799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:51.280813 17799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:51.280895 17799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:51.280956 17799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.1
I20260812 19:07:51.285983 17799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:45029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.130.1
--webserver_port=0
--tserver_master_addrs=127.25.130.62:33101
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.1
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:51.287139 17799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:51.288630 17799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:51.298861 17807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:51.298864 17805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:51.298863 17804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:51.299346 17799 server_base.cc:1034] running on GCE node
I20260812 19:07:51.299784 17799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:51.300460 17799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:51.301620 17799 hybrid_clock.cc:648] HybridClock initialized: now 1786561671301599 us; error 33 us; skew 500 ppm
I20260812 19:07:51.303571 17799 webserver.cc:460] Webserver started at http://127.25.130.1:36303/ using document root <none> and password file <none>
I20260812 19:07:51.304229 17799 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:51.304307 17799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:51.304522 17799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:51.306722 17799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-0/data/instance:
uuid: "512c19c07beb423a96942f1b71589100"
format_stamp: "Formatted at 2026-08-12 19:07:51 on dist-test-slave-49h3"
server_key: "3ba2763f7ffcf3e508a6125d800c69d1"
server_key_iv: "ad29121033b385a163eed48d8dc48eff"
server_key_version: "encryptionkey@0"
I20260812 19:07:51.307307 17799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance:
uuid: "512c19c07beb423a96942f1b71589100"
format_stamp: "Formatted at 2026-08-12 19:07:51 on dist-test-slave-49h3"
server_key: "3ba2763f7ffcf3e508a6125d800c69d1"
server_key_iv: "ad29121033b385a163eed48d8dc48eff"
server_key_version: "encryptionkey@0"
I20260812 19:07:51.311193 17799 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 19:07:51.313707 17813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:51.314956 17799 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 19:07:51.315078 17799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-0/wal
uuid: "512c19c07beb423a96942f1b71589100"
format_stamp: "Formatted at 2026-08-12 19:07:51 on dist-test-slave-49h3"
server_key: "3ba2763f7ffcf3e508a6125d800c69d1"
server_key_iv: "ad29121033b385a163eed48d8dc48eff"
server_key_version: "encryptionkey@0"
I20260812 19:07:51.315191 17799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:51.340867 17799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:51.341650 17799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:51.341830 17799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:51.342440 17799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:07:51.343454 17799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:07:51.343508 17799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:51.343551 17799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:07:51.343565 17799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:51.352471 17799 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.1:34443
I20260812 19:07:51.352523 17946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.1:34443 every 8 connection(s)
I20260812 19:07:51.353541 17799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-0/data/info.pb
I20260812 19:07:51.359095 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 17799
I20260812 19:07:51.359230 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-0/wal/instance
I20260812 19:07:51.359529 26120 external_mini_cluster.cc:1428] Setting key 11885c1555d6d9cf228c3877aa2643fb
I20260812 19:07:51.361539 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.2:0
--local_ip_for_outbound_sockets=127.25.130.2
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:33101
--builtin_ntp_servers=127.25.130.20:45029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 19:07:51.363168 17775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:51.355533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:52361 (local address 127.25.130.62:33101)
0812 19:07:51.355668 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:51.355671 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:51.356788 (+ 1117us) server_negotiation.cc:407] Connection header received
0812 19:07:51.358206 (+ 1418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:51.358210 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:51.358269 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:51.358343 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:51.359298 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.359898 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.361168 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.361393 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.361520 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:51.362660 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:51.362663 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:51.362665 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:07:51.362678 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:51.362700 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:07:51.362708 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:51.362769 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:51.362919 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 19:07:51.363013 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 19:07:51.364091 17949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:51.355925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33101 (local address 127.25.130.1:52361)
0812 19:07:51.356637 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:51.356681 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:51.357984 (+ 1303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:51.358504 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:51.358511 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:51.358600 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:07:51.359120 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:51.359134 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.360009 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.360012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:51.361039 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:51.361050 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.361566 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.361571 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:51.361782 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:51.362385 (+ 603us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:07:51.362386 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:07:51.362430 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:07:51.362501 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:07:51.362780 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:07:51.362790 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:07:51.362796 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 19:07:51.362909 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 19:07:51.363224 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":122,"threads_started":1}
I20260812 19:07:51.365370 17947 heartbeater.cc:347] Connected to a master server at 127.25.130.62:33101
I20260812 19:07:51.365648 17947 heartbeater.cc:464] Registering TS with master...
I20260812 19:07:51.366256 17947 heartbeater.cc:511] Master 127.25.130.62:33101 requested a full tablet report, sending...
I20260812 19:07:51.368044 17723 ts_manager.cc:194] Registered new tserver with Master: 512c19c07beb423a96942f1b71589100 (127.25.130.1:34443)
I20260812 19:07:51.369784 17723 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.130.1:52361
W20260812 19:07:51.475898 17950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:51.476199 17950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:51.476238 17950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:51.480324 17950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:51.480398 17950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:51.480473 17950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.2
I20260812 19:07:51.487041 17950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:45029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.130.2
--webserver_port=0
--tserver_master_addrs=127.25.130.62:33101
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.2
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:51.488173 17950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:51.489593 17950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:51.499225 17961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:51.499274 17957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:51.499374 17950 server_base.cc:1034] running on GCE node
W20260812 19:07:51.499550 17958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:51.499965 17950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:51.500615 17950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:51.501778 17950 hybrid_clock.cc:648] HybridClock initialized: now 1786561671501755 us; error 32 us; skew 500 ppm
I20260812 19:07:51.503890 17950 webserver.cc:460] Webserver started at http://127.25.130.2:43509/ using document root <none> and password file <none>
I20260812 19:07:51.504575 17950 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:51.504669 17950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:51.504931 17950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:51.507318 17950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-1/data/instance:
uuid: "279f8dbdfe22464a99e23b1b3b3d798b"
format_stamp: "Formatted at 2026-08-12 19:07:51 on dist-test-slave-49h3"
server_key: "27786ce95c38a07d0de9c362bab393f9"
server_key_iv: "c9ce2062cf3a2dd887883782f4e985a9"
server_key_version: "encryptionkey@0"
I20260812 19:07:51.507968 17950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance:
uuid: "279f8dbdfe22464a99e23b1b3b3d798b"
format_stamp: "Formatted at 2026-08-12 19:07:51 on dist-test-slave-49h3"
server_key: "27786ce95c38a07d0de9c362bab393f9"
server_key_iv: "c9ce2062cf3a2dd887883782f4e985a9"
server_key_version: "encryptionkey@0"
I20260812 19:07:51.512212 17950 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 19:07:51.515331 17969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:51.516908 17950 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 19:07:51.517058 17950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-1/wal
uuid: "279f8dbdfe22464a99e23b1b3b3d798b"
format_stamp: "Formatted at 2026-08-12 19:07:51 on dist-test-slave-49h3"
server_key: "27786ce95c38a07d0de9c362bab393f9"
server_key_iv: "c9ce2062cf3a2dd887883782f4e985a9"
server_key_version: "encryptionkey@0"
I20260812 19:07:51.517200 17950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:51.547951 17950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:51.548768 17950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:51.548970 17950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:51.549670 17950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:07:51.550742 17950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:07:51.550796 17950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:51.550853 17950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:07:51.550879 17950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:51.560055 17950 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.2:35835
I20260812 19:07:51.560106 18107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.2:35835 every 8 connection(s)
I20260812 19:07:51.561077 17950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-1/data/info.pb
I20260812 19:07:51.569250 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 17950
I20260812 19:07:51.569378 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-1/wal/instance
I20260812 19:07:51.569689 26120 external_mini_cluster.cc:1428] Setting key 0d5246c376128a5727c3e9489099b9d3
I20260812 19:07:51.571771 17775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:51.563243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.2:58071 (local address 127.25.130.62:33101)
0812 19:07:51.563417 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:51.563421 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:51.564944 (+ 1523us) server_negotiation.cc:407] Connection header received
0812 19:07:51.566709 (+ 1765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:51.566713 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:51.566782 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:51.566861 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:51.567866 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.568402 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.569638 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.569916 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.570009 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:51.571298 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:51.571301 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:51.571303 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:07:51.571315 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:51.571337 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:07:51.571345 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:51.571403 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:51.571579 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 19:07:51.571641 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 19:07:51.571805 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.130.3:0
--local_ip_for_outbound_sockets=127.25.130.3
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:33101
--builtin_ntp_servers=127.25.130.20:45029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 19:07:51.572999 18110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:51.563786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33101 (local address 127.25.130.2:58071)
0812 19:07:51.564758 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:51.564826 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:51.566457 (+ 1631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:51.567070 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:51.567078 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:51.567175 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:07:51.567712 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:51.567727 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.568535 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.568538 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:51.569505 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:51.569519 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.570084 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.570089 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:51.570289 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:51.570984 (+ 695us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:07:51.570986 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:07:51.571051 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:07:51.571141 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:07:51.571426 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:07:51.571435 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:07:51.571442 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 19:07:51.571590 (+ 148us) client_negotiation.cc:240] Negotiation successful
0812 19:07:51.571920 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":170,"threads_started":1}
I20260812 19:07:51.574335 18108 heartbeater.cc:347] Connected to a master server at 127.25.130.62:33101
I20260812 19:07:51.574610 18108 heartbeater.cc:464] Registering TS with master...
I20260812 19:07:51.575160 18108 heartbeater.cc:511] Master 127.25.130.62:33101 requested a full tablet report, sending...
I20260812 19:07:51.576272 17723 ts_manager.cc:194] Registered new tserver with Master: 279f8dbdfe22464a99e23b1b3b3d798b (127.25.130.2:35835)
I20260812 19:07:51.576905 17723 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.130.2:58071
W20260812 19:07:51.686633 18111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:51.686930 18111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:51.686971 18111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:51.691107 18111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:51.691174 18111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:51.691247 18111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.130.3
I20260812 19:07:51.696362 18111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:45029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.130.3
--webserver_port=0
--tserver_master_addrs=127.25.130.62:33101
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.130.3
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:51.697515 18111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:51.698956 18111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:51.708604 18117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:51.708599 18119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:51.708599 18116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:51.708664 18111 server_base.cc:1034] running on GCE node
I20260812 19:07:51.709314 18111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:51.709988 18111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:51.711164 18111 hybrid_clock.cc:648] HybridClock initialized: now 1786561671711144 us; error 36 us; skew 500 ppm
I20260812 19:07:51.712985 18111 webserver.cc:460] Webserver started at http://127.25.130.3:34015/ using document root <none> and password file <none>
I20260812 19:07:51.713676 18111 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:51.713757 18111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:51.713943 18111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:51.716012 18111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-2/data/instance:
uuid: "5e24f714afcb4bb8bec87f75aae157ce"
format_stamp: "Formatted at 2026-08-12 19:07:51 on dist-test-slave-49h3"
server_key: "1f5c30a19370fb508aac78ee9e4ac919"
server_key_iv: "33d10f0f25fb9eb155c8f3f138acc48d"
server_key_version: "encryptionkey@0"
I20260812 19:07:51.716555 18111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance:
uuid: "5e24f714afcb4bb8bec87f75aae157ce"
format_stamp: "Formatted at 2026-08-12 19:07:51 on dist-test-slave-49h3"
server_key: "1f5c30a19370fb508aac78ee9e4ac919"
server_key_iv: "33d10f0f25fb9eb155c8f3f138acc48d"
server_key_version: "encryptionkey@0"
I20260812 19:07:51.720299 18111 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 19:07:51.722841 18125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:51.724107 18111 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 19:07:51.724224 18111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-2/wal
uuid: "5e24f714afcb4bb8bec87f75aae157ce"
format_stamp: "Formatted at 2026-08-12 19:07:51 on dist-test-slave-49h3"
server_key: "1f5c30a19370fb508aac78ee9e4ac919"
server_key_iv: "33d10f0f25fb9eb155c8f3f138acc48d"
server_key_version: "encryptionkey@0"
I20260812 19:07:51.724329 18111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:51.748252 18111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:51.748987 18111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:51.749179 18111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:51.749856 18111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:07:51.750849 18111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:07:51.750898 18111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:51.750942 18111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:07:51.750972 18111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:51.759507 18111 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.3:39955
I20260812 19:07:51.759547 18265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.3:39955 every 8 connection(s)
I20260812 19:07:51.760499 18111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-2/data/info.pb
I20260812 19:07:51.769878 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 18111
I20260812 19:07:51.770004 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-2/wal/instance
I20260812 19:07:51.770017 17775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:51.762470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.3:58869 (local address 127.25.130.62:33101)
0812 19:07:51.762651 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:51.762656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:51.763800 (+ 1144us) server_negotiation.cc:407] Connection header received
0812 19:07:51.765189 (+ 1389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:51.765192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:51.765246 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:51.765322 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:51.766308 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.766843 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.768009 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.768255 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.768351 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:51.769518 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:51.769521 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:51.769524 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:07:51.769539 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:51.769565 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:07:51.769576 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:51.769651 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:51.769774 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 19:07:51.769849 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 19:07:51.770319 26120 external_mini_cluster.cc:1428] Setting key 35761a8bb95ad17aa08652c4b460e333
I20260812 19:07:51.771049 18268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:51.762914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:33101 (local address 127.25.130.3:58869)
0812 19:07:51.763641 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:51.763687 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:51.764980 (+ 1293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:51.765514 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:51.765521 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:51.765621 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:07:51.766165 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:51.766180 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.766949 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.766952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:51.767867 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:51.767878 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.768408 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.768411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:51.768562 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:51.769218 (+ 656us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:07:51.769220 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:07:51.769264 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:07:51.769337 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:07:51.769651 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:07:51.769656 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:07:51.769660 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 19:07:51.769754 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 19:07:51.770024 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":124,"threads_started":1}
I20260812 19:07:51.772322 18266 heartbeater.cc:347] Connected to a master server at 127.25.130.62:33101
I20260812 19:07:51.772586 18266 heartbeater.cc:464] Registering TS with master...
I20260812 19:07:51.773150 18266 heartbeater.cc:511] Master 127.25.130.62:33101 requested a full tablet report, sending...
I20260812 19:07:51.774339 17721 ts_manager.cc:194] Registered new tserver with Master: 5e24f714afcb4bb8bec87f75aae157ce (127.25.130.3:39955)
I20260812 19:07:51.774948 17721 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.130.3:58869
I20260812 19:07:51.784399 26120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:07:51.790217 17775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:51.786454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47460 (local address 127.25.130.62:33101)
0812 19:07:51.786589 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:51.786594 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:51.786768 (+ 174us) server_negotiation.cc:407] Connection header received
0812 19:07:51.786886 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:51.786890 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:51.786938 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:51.787005 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:51.787319 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.787845 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.788682 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.788898 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.788986 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:51.789808 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:51.789812 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:51.789814 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:07:51.789828 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:51.789848 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:07:51.789856 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:51.789916 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:51.790002 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 19:07:51.790060 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 19:07:51.793265 17721 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47460:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:07:51.796535 17721 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:07:51.810305 18279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:51.805107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:35835 (local address 127.0.0.1:56502)
0812 19:07:51.805556 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:51.805615 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:51.805748 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:51.806437 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:51.806444 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:51.806470 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:51.806637 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:51.806645 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.807843 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.807846 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:51.808717 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:51.808728 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.809747 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.809751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:51.809880 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:51.809933 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 19:07:51.810016 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 19:07:51.810137 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":69,"threads_started":1}
I20260812 19:07:51.811307 18280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:51.805239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56502 (local address 127.25.130.2:35835)
0812 19:07:51.805899 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:51.805914 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:51.805939 (+ 25us) server_negotiation.cc:407] Connection header received
0812 19:07:51.806038 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:51.806043 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:51.806257 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:51.806419 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:51.806781 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.807750 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.808847 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.809614 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.810774 (+ 1160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:51.810986 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:51.811040 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 19:07:51.811140 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":198,"threads_started":1}
I20260812 19:07:51.811420 18282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:51.805866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:39955 (local address 127.0.0.1:43268)
0812 19:07:51.806671 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:51.806683 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:51.806761 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:51.807449 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:51.807452 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:51.807472 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:51.807565 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:51.807571 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.808904 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.808907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:51.810054 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:51.810061 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.811080 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.811083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:51.811154 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:51.811174 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 19:07:51.811213 (+ 39us) client_negotiation.cc:240] Negotiation successful
0812 19:07:51.811267 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":746,"thread_start_us":91,"threads_started":1}
I20260812 19:07:51.811439 18283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:51.805940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42110 (local address 127.25.130.1:34443)
0812 19:07:51.806638 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:51.806649 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:51.806672 (+ 23us) server_negotiation.cc:407] Connection header received
0812 19:07:51.806752 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:51.806757 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:51.806945 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:51.807367 (+ 422us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:51.807510 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.808643 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.810121 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.810879 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.810983 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:51.811164 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:51.811210 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 19:07:51.811295 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":116,"threads_started":1}
I20260812 19:07:51.811854 18284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:51.806226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43268 (local address 127.25.130.3:39955)
0812 19:07:51.807010 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:51.807033 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:51.807057 (+ 24us) server_negotiation.cc:407] Connection header received
0812 19:07:51.807127 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:51.807132 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:51.807307 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:51.807441 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:51.807681 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.808761 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.810279 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.810942 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.811344 (+ 402us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:51.811526 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:51.811580 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 19:07:51.811689 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":149,"threads_started":1}
I20260812 19:07:51.812088 18277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:51.805148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:34443 (local address 127.0.0.1:42110)
0812 19:07:51.806140 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:51.806153 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:51.806231 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:51.807101 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:51.807104 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:51.807122 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:51.807225 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:51.807232 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.808765 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.808768 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:51.809987 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:51.809998 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.811004 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.811008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:51.811103 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:51.811131 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 19:07:51.811189 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 19:07:51.811257 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":931,"thread_start_us":115,"threads_started":1}
I20260812 19:07:51.814688 17869 tablet_service.cc:1467] Processing CreateTablet for tablet d5f496d4092e487c82b9cc8b31b7f0c6 (DEFAULT_TABLE table=test-table [id=bc50c1d152764798b74af7b9c32d348e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:07:51.814702 18023 tablet_service.cc:1467] Processing CreateTablet for tablet d5f496d4092e487c82b9cc8b31b7f0c6 (DEFAULT_TABLE table=test-table [id=bc50c1d152764798b74af7b9c32d348e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:07:51.814702 18180 tablet_service.cc:1467] Processing CreateTablet for tablet d5f496d4092e487c82b9cc8b31b7f0c6 (DEFAULT_TABLE table=test-table [id=bc50c1d152764798b74af7b9c32d348e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:07:51.815692 17869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5f496d4092e487c82b9cc8b31b7f0c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:51.815742 18180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5f496d4092e487c82b9cc8b31b7f0c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:51.815743 18023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5f496d4092e487c82b9cc8b31b7f0c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:51.822688 18285 tablet_bootstrap.cc:492] T d5f496d4092e487c82b9cc8b31b7f0c6 P 512c19c07beb423a96942f1b71589100: Bootstrap starting.
I20260812 19:07:51.823663 18287 tablet_bootstrap.cc:492] T d5f496d4092e487c82b9cc8b31b7f0c6 P 279f8dbdfe22464a99e23b1b3b3d798b: Bootstrap starting.
I20260812 19:07:51.823663 18286 tablet_bootstrap.cc:492] T d5f496d4092e487c82b9cc8b31b7f0c6 P 5e24f714afcb4bb8bec87f75aae157ce: Bootstrap starting.
I20260812 19:07:51.825242 18285 tablet_bootstrap.cc:654] T d5f496d4092e487c82b9cc8b31b7f0c6 P 512c19c07beb423a96942f1b71589100: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:51.825851 18286 tablet_bootstrap.cc:654] T d5f496d4092e487c82b9cc8b31b7f0c6 P 5e24f714afcb4bb8bec87f75aae157ce: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:51.825852 18287 tablet_bootstrap.cc:654] T d5f496d4092e487c82b9cc8b31b7f0c6 P 279f8dbdfe22464a99e23b1b3b3d798b: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:51.826073 18285 log.cc:824] T d5f496d4092e487c82b9cc8b31b7f0c6 P 512c19c07beb423a96942f1b71589100: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:51.826719 18287 log.cc:824] T d5f496d4092e487c82b9cc8b31b7f0c6 P 279f8dbdfe22464a99e23b1b3b3d798b: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:51.826719 18286 log.cc:824] T d5f496d4092e487c82b9cc8b31b7f0c6 P 5e24f714afcb4bb8bec87f75aae157ce: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:51.828572 18285 tablet_bootstrap.cc:492] T d5f496d4092e487c82b9cc8b31b7f0c6 P 512c19c07beb423a96942f1b71589100: No bootstrap required, opened a new log
I20260812 19:07:51.828742 18286 tablet_bootstrap.cc:492] T d5f496d4092e487c82b9cc8b31b7f0c6 P 5e24f714afcb4bb8bec87f75aae157ce: No bootstrap required, opened a new log
I20260812 19:07:51.828832 18285 ts_tablet_manager.cc:1397] T d5f496d4092e487c82b9cc8b31b7f0c6 P 512c19c07beb423a96942f1b71589100: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 19:07:51.828886 18287 tablet_bootstrap.cc:492] T d5f496d4092e487c82b9cc8b31b7f0c6 P 279f8dbdfe22464a99e23b1b3b3d798b: No bootstrap required, opened a new log
I20260812 19:07:51.828963 18286 ts_tablet_manager.cc:1397] T d5f496d4092e487c82b9cc8b31b7f0c6 P 5e24f714afcb4bb8bec87f75aae157ce: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260812 19:07:51.829087 18287 ts_tablet_manager.cc:1397] T d5f496d4092e487c82b9cc8b31b7f0c6 P 279f8dbdfe22464a99e23b1b3b3d798b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 19:07:51.832311 18287 raft_consensus.cc:348] T d5f496d4092e487c82b9cc8b31b7f0c6 P 279f8dbdfe22464a99e23b1b3b3d798b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279f8dbdfe22464a99e23b1b3b3d798b" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 35835 } } peers { permanent_uuid: "512c19c07beb423a96942f1b71589100" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 34443 } } peers { permanent_uuid: "5e24f714afcb4bb8bec87f75aae157ce" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 39955 } }
I20260812 19:07:51.832563 18287 raft_consensus.cc:374] T d5f496d4092e487c82b9cc8b31b7f0c6 P 279f8dbdfe22464a99e23b1b3b3d798b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:51.832638 18287 raft_consensus.cc:729] T d5f496d4092e487c82b9cc8b31b7f0c6 P 279f8dbdfe22464a99e23b1b3b3d798b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 279f8dbdfe22464a99e23b1b3b3d798b, State: Initialized, Role: FOLLOWER
I20260812 19:07:51.833098 18287 consensus_queue.cc:260] T d5f496d4092e487c82b9cc8b31b7f0c6 P 279f8dbdfe22464a99e23b1b3b3d798b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279f8dbdfe22464a99e23b1b3b3d798b" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 35835 } } peers { permanent_uuid: "512c19c07beb423a96942f1b71589100" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 34443 } } peers { permanent_uuid: "5e24f714afcb4bb8bec87f75aae157ce" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 39955 } }
I20260812 19:07:51.833154 18285 raft_consensus.cc:348] T d5f496d4092e487c82b9cc8b31b7f0c6 P 512c19c07beb423a96942f1b71589100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279f8dbdfe22464a99e23b1b3b3d798b" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 35835 } } peers { permanent_uuid: "512c19c07beb423a96942f1b71589100" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 34443 } } peers { permanent_uuid: "5e24f714afcb4bb8bec87f75aae157ce" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 39955 } }
I20260812 19:07:51.833153 18286 raft_consensus.cc:348] T d5f496d4092e487c82b9cc8b31b7f0c6 P 5e24f714afcb4bb8bec87f75aae157ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279f8dbdfe22464a99e23b1b3b3d798b" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 35835 } } peers { permanent_uuid: "512c19c07beb423a96942f1b71589100" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 34443 } } peers { permanent_uuid: "5e24f714afcb4bb8bec87f75aae157ce" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 39955 } }
I20260812 19:07:51.833413 18285 raft_consensus.cc:374] T d5f496d4092e487c82b9cc8b31b7f0c6 P 512c19c07beb423a96942f1b71589100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:51.833413 18286 raft_consensus.cc:374] T d5f496d4092e487c82b9cc8b31b7f0c6 P 5e24f714afcb4bb8bec87f75aae157ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:51.833495 18285 raft_consensus.cc:729] T d5f496d4092e487c82b9cc8b31b7f0c6 P 512c19c07beb423a96942f1b71589100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 512c19c07beb423a96942f1b71589100, State: Initialized, Role: FOLLOWER
I20260812 19:07:51.833503 18286 raft_consensus.cc:729] T d5f496d4092e487c82b9cc8b31b7f0c6 P 5e24f714afcb4bb8bec87f75aae157ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e24f714afcb4bb8bec87f75aae157ce, State: Initialized, Role: FOLLOWER
I20260812 19:07:51.833948 18286 consensus_queue.cc:260] T d5f496d4092e487c82b9cc8b31b7f0c6 P 5e24f714afcb4bb8bec87f75aae157ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279f8dbdfe22464a99e23b1b3b3d798b" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 35835 } } peers { permanent_uuid: "512c19c07beb423a96942f1b71589100" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 34443 } } peers { permanent_uuid: "5e24f714afcb4bb8bec87f75aae157ce" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 39955 } }
I20260812 19:07:51.833948 18285 consensus_queue.cc:260] T d5f496d4092e487c82b9cc8b31b7f0c6 P 512c19c07beb423a96942f1b71589100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279f8dbdfe22464a99e23b1b3b3d798b" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 35835 } } peers { permanent_uuid: "512c19c07beb423a96942f1b71589100" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 34443 } } peers { permanent_uuid: "5e24f714afcb4bb8bec87f75aae157ce" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 39955 } }
I20260812 19:07:51.834208 18287 ts_tablet_manager.cc:1428] T d5f496d4092e487c82b9cc8b31b7f0c6 P 279f8dbdfe22464a99e23b1b3b3d798b: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260812 19:07:51.834590 18108 heartbeater.cc:503] Master 127.25.130.62:33101 was elected leader, sending a full tablet report...
I20260812 19:07:51.834947 18286 ts_tablet_manager.cc:1428] T d5f496d4092e487c82b9cc8b31b7f0c6 P 5e24f714afcb4bb8bec87f75aae157ce: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260812 19:07:51.835439 18285 ts_tablet_manager.cc:1428] T d5f496d4092e487c82b9cc8b31b7f0c6 P 512c19c07beb423a96942f1b71589100: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 19:07:51.835534 17947 heartbeater.cc:503] Master 127.25.130.62:33101 was elected leader, sending a full tablet report...
I20260812 19:07:51.835547 18266 heartbeater.cc:503] Master 127.25.130.62:33101 was elected leader, sending a full tablet report...
W20260812 19:07:51.854812 17948 tablet.cc:2369] T d5f496d4092e487c82b9cc8b31b7f0c6 P 512c19c07beb423a96942f1b71589100: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:07:51.994892 18292 raft_consensus.cc:482] T d5f496d4092e487c82b9cc8b31b7f0c6 P 512c19c07beb423a96942f1b71589100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:51.995121 18292 raft_consensus.cc:504] T d5f496d4092e487c82b9cc8b31b7f0c6 P 512c19c07beb423a96942f1b71589100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279f8dbdfe22464a99e23b1b3b3d798b" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 35835 } } peers { permanent_uuid: "512c19c07beb423a96942f1b71589100" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 34443 } } peers { permanent_uuid: "5e24f714afcb4bb8bec87f75aae157ce" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 39955 } }
I20260812 19:07:51.996263 18292 leader_election.cc:288] T d5f496d4092e487c82b9cc8b31b7f0c6 P 512c19c07beb423a96942f1b71589100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 279f8dbdfe22464a99e23b1b3b3d798b (127.25.130.2:35835), 5e24f714afcb4bb8bec87f75aae157ce (127.25.130.3:39955)
I20260812 19:07:52.000002 18280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:51.996658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:40439 (local address 127.25.130.2:35835)
0812 19:07:51.996789 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:51.996793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:51.996977 (+ 184us) server_negotiation.cc:407] Connection header received
0812 19:07:51.997057 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:51.997061 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:51.997115 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:51.997206 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:51.997507 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.998017 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.999087 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.999483 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.999597 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:51.999650 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:51.999810 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 19:07:51.999884 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 19:07:52.000020 18294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:51.996528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.2:35835 (local address 127.25.130.1:40439)
0812 19:07:51.996821 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:51.996836 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:51.996950 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:51.997240 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:51.997243 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:51.997265 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:51.997383 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:51.997387 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.998128 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.998132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:51.998960 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:51.998970 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.999594 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.999597 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:51.999697 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:51.999719 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 19:07:51.999784 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 19:07:51.999859 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":114,"threads_started":1}
I20260812 19:07:52.000447 18295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:51.996966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.3:39955 (local address 127.25.130.1:39659)
0812 19:07:51.997331 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:51.997345 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:07:51.997431 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:07:51.997759 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:07:51.997762 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:07:51.997779 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:07:51.997898 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:51.997903 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.998613 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.998617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:51.999368 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:07:51.999377 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:07:52.000014 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:07:52.000018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:07:52.000127 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:52.000155 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 19:07:52.000235 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 19:07:52.000304 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":123,"threads_started":1}
I20260812 19:07:52.000471 18284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:51.997408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.130.1:39659 (local address 127.25.130.3:39955)
0812 19:07:51.997564 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:51.997568 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:51.997581 (+ 13us) server_negotiation.cc:407] Connection header received
0812 19:07:51.997629 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:51.997631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:51.997667 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:51.997728 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:07:51.998006 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.998501 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:51.999487 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:51.999898 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:52.000003 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:52.000067 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:52.000279 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 19:07:52.000357 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 19:07:52.000802 18052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5f496d4092e487c82b9cc8b31b7f0c6" candidate_uuid: "512c19c07beb423a96942f1b71589100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "279f8dbdfe22464a99e23b1b3b3d798b" is_pre_election: true
I20260812 19:07:52.001120 18052 raft_consensus.cc:2454] T d5f496d4092e487c82b9cc8b31b7f0c6 P 279f8dbdfe22464a99e23b1b3b3d798b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 512c19c07beb423a96942f1b71589100 in term 0.
I20260812 19:07:52.001163 18206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5f496d4092e487c82b9cc8b31b7f0c6" candidate_uuid: "512c19c07beb423a96942f1b71589100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e24f714afcb4bb8bec87f75aae157ce" is_pre_election: true
I20260812 19:07:52.001426 18206 raft_consensus.cc:2454] T d5f496d4092e487c82b9cc8b31b7f0c6 P 5e24f714afcb4bb8bec87f75aae157ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 512c19c07beb423a96942f1b71589100 in term 0.
I20260812 19:07:52.001619 17815 leader_election.cc:302] T d5f496d4092e487c82b9cc8b31b7f0c6 P 512c19c07beb423a96942f1b71589100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 279f8dbdfe22464a99e23b1b3b3d798b, 512c19c07beb423a96942f1b71589100; no voters:
I20260812 19:07:52.001910 18292 raft_consensus.cc:2781] T d5f496d4092e487c82b9cc8b31b7f0c6 P 512c19c07beb423a96942f1b71589100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:07:52.001995 18292 raft_consensus.cc:482] T d5f496d4092e487c82b9cc8b31b7f0c6 P 512c19c07beb423a96942f1b71589100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:52.002023 18292 raft_consensus.cc:3037] T d5f496d4092e487c82b9cc8b31b7f0c6 P 512c19c07beb423a96942f1b71589100 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:52.003073 18292 raft_consensus.cc:504] T d5f496d4092e487c82b9cc8b31b7f0c6 P 512c19c07beb423a96942f1b71589100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279f8dbdfe22464a99e23b1b3b3d798b" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 35835 } } peers { permanent_uuid: "512c19c07beb423a96942f1b71589100" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 34443 } } peers { permanent_uuid: "5e24f714afcb4bb8bec87f75aae157ce" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 39955 } }
I20260812 19:07:52.003417 18292 leader_election.cc:288] T d5f496d4092e487c82b9cc8b31b7f0c6 P 512c19c07beb423a96942f1b71589100 [CANDIDATE]: Term 1 election: Requested vote from peers 279f8dbdfe22464a99e23b1b3b3d798b (127.25.130.2:35835), 5e24f714afcb4bb8bec87f75aae157ce (127.25.130.3:39955)
I20260812 19:07:52.003815 18052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5f496d4092e487c82b9cc8b31b7f0c6" candidate_uuid: "512c19c07beb423a96942f1b71589100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "279f8dbdfe22464a99e23b1b3b3d798b"
I20260812 19:07:52.003849 18206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5f496d4092e487c82b9cc8b31b7f0c6" candidate_uuid: "512c19c07beb423a96942f1b71589100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e24f714afcb4bb8bec87f75aae157ce"
I20260812 19:07:52.003953 18052 raft_consensus.cc:3037] T d5f496d4092e487c82b9cc8b31b7f0c6 P 279f8dbdfe22464a99e23b1b3b3d798b [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:52.003953 18206 raft_consensus.cc:3037] T d5f496d4092e487c82b9cc8b31b7f0c6 P 5e24f714afcb4bb8bec87f75aae157ce [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:52.005218 18206 raft_consensus.cc:2454] T d5f496d4092e487c82b9cc8b31b7f0c6 P 5e24f714afcb4bb8bec87f75aae157ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 512c19c07beb423a96942f1b71589100 in term 1.
I20260812 19:07:52.005218 18052 raft_consensus.cc:2454] T d5f496d4092e487c82b9cc8b31b7f0c6 P 279f8dbdfe22464a99e23b1b3b3d798b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 512c19c07beb423a96942f1b71589100 in term 1.
I20260812 19:07:52.005560 17815 leader_election.cc:302] T d5f496d4092e487c82b9cc8b31b7f0c6 P 512c19c07beb423a96942f1b71589100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 279f8dbdfe22464a99e23b1b3b3d798b, 512c19c07beb423a96942f1b71589100; no voters:
I20260812 19:07:52.005744 18292 raft_consensus.cc:2781] T d5f496d4092e487c82b9cc8b31b7f0c6 P 512c19c07beb423a96942f1b71589100 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:07:52.005965 18292 raft_consensus.cc:686] T d5f496d4092e487c82b9cc8b31b7f0c6 P 512c19c07beb423a96942f1b71589100 [term 1 LEADER]: Becoming Leader. State: Replica: 512c19c07beb423a96942f1b71589100, State: Running, Role: LEADER
I20260812 19:07:52.006289 18292 consensus_queue.cc:237] T d5f496d4092e487c82b9cc8b31b7f0c6 P 512c19c07beb423a96942f1b71589100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279f8dbdfe22464a99e23b1b3b3d798b" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 35835 } } peers { permanent_uuid: "512c19c07beb423a96942f1b71589100" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 34443 } } peers { permanent_uuid: "5e24f714afcb4bb8bec87f75aae157ce" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 39955 } }
I20260812 19:07:52.009567 17723 catalog_manager.cc:5495] T d5f496d4092e487c82b9cc8b31b7f0c6 P 512c19c07beb423a96942f1b71589100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 512c19c07beb423a96942f1b71589100 (127.25.130.1). New cstate: current_term: 1 leader_uuid: "512c19c07beb423a96942f1b71589100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279f8dbdfe22464a99e23b1b3b3d798b" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 35835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "512c19c07beb423a96942f1b71589100" member_type: VOTER last_known_addr { host: "127.25.130.1" port: 34443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e24f714afcb4bb8bec87f75aae157ce" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 39955 } health_report { overall_health: UNKNOWN } } }
W20260812 19:07:52.011477 18267 tablet.cc:2369] T d5f496d4092e487c82b9cc8b31b7f0c6 P 5e24f714afcb4bb8bec87f75aae157ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:07:52.048651 18283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:52.045233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42126 (local address 127.25.130.1:34443)
0812 19:07:52.045360 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:52.045364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:52.045376 (+ 12us) server_negotiation.cc:407] Connection header received
0812 19:07:52.045498 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:52.045502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:52.045570 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:52.045661 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:07:52.045868 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:52.046396 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:52.047187 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:52.047398 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:52.047468 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:52.047733 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:07:52.048178 (+ 445us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:07:52.048279 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:52.048439 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 19:07:52.048498 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 19:07:52.054903 18052 raft_consensus.cc:1261] T d5f496d4092e487c82b9cc8b31b7f0c6 P 279f8dbdfe22464a99e23b1b3b3d798b [term 1 FOLLOWER]: Refusing update from remote peer 512c19c07beb423a96942f1b71589100: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:52.054903 18206 raft_consensus.cc:1261] T d5f496d4092e487c82b9cc8b31b7f0c6 P 5e24f714afcb4bb8bec87f75aae157ce [term 1 FOLLOWER]: Refusing update from remote peer 512c19c07beb423a96942f1b71589100: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:52.055629 18296 consensus_queue.cc:1035] T d5f496d4092e487c82b9cc8b31b7f0c6 P 512c19c07beb423a96942f1b71589100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e24f714afcb4bb8bec87f75aae157ce" member_type: VOTER last_known_addr { host: "127.25.130.3" port: 39955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:07:52.055833 18292 consensus_queue.cc:1035] T d5f496d4092e487c82b9cc8b31b7f0c6 P 512c19c07beb423a96942f1b71589100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "279f8dbdfe22464a99e23b1b3b3d798b" member_type: VOTER last_known_addr { host: "127.25.130.2" port: 35835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 19:07:52.062225 18109 tablet.cc:2369] T d5f496d4092e487c82b9cc8b31b7f0c6 P 279f8dbdfe22464a99e23b1b3b3d798b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:07:52.063134 18299 mvcc.cc:205] Tried to move back new op lower bound from 7317756608729055232 to 7317756608540209152. Current Snapshot: MvccSnapshot[applied={T|T < 7317756608729055232}]
I20260812 19:07:52.063798 18301 mvcc.cc:205] Tried to move back new op lower bound from 7317756608729055232 to 7317756608540209152. Current Snapshot: MvccSnapshot[applied={T|T < 7317756608729055232}]
I20260812 19:07:52.065392 18300 mvcc.cc:205] Tried to move back new op lower bound from 7317756608729055232 to 7317756608540209152. Current Snapshot: MvccSnapshot[applied={T|T < 7317756608729055232}]
I20260812 19:07:52.072156 17720 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:07:52.072314 17720 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:07:52.074692 17720 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c5af1c52ad0463991e503aa9435df70: Sending DeleteTablet for 3 replicas of tablet d5f496d4092e487c82b9cc8b31b7f0c6
I20260812 19:07:52.075537 17869 tablet_service.cc:1514] Processing DeleteTablet for tablet d5f496d4092e487c82b9cc8b31b7f0c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:07:52 UTC) from {username='slave'} at 127.0.0.1:42110
I20260812 19:07:52.075695 18023 tablet_service.cc:1514] Processing DeleteTablet for tablet d5f496d4092e487c82b9cc8b31b7f0c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:07:52 UTC) from {username='slave'} at 127.0.0.1:56502
I20260812 19:07:52.076114 18180 tablet_service.cc:1514] Processing DeleteTablet for tablet d5f496d4092e487c82b9cc8b31b7f0c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:07:52 UTC) from {username='slave'} at 127.0.0.1:43268
I20260812 19:07:52.076574 18311 tablet_replica.cc:330] T d5f496d4092e487c82b9cc8b31b7f0c6 P 5e24f714afcb4bb8bec87f75aae157ce: stopping tablet replica
I20260812 19:07:52.076960 18311 raft_consensus.cc:2229] T d5f496d4092e487c82b9cc8b31b7f0c6 P 5e24f714afcb4bb8bec87f75aae157ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:52.077279 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 17799
I20260812 19:07:52.077361 18311 raft_consensus.cc:2258] T d5f496d4092e487c82b9cc8b31b7f0c6 P 5e24f714afcb4bb8bec87f75aae157ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:52.079286 18310 tablet_replica.cc:330] T d5f496d4092e487c82b9cc8b31b7f0c6 P 279f8dbdfe22464a99e23b1b3b3d798b: stopping tablet replica
I20260812 19:07:52.079610 18310 raft_consensus.cc:2229] T d5f496d4092e487c82b9cc8b31b7f0c6 P 279f8dbdfe22464a99e23b1b3b3d798b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:52.079908 18310 raft_consensus.cc:2258] T d5f496d4092e487c82b9cc8b31b7f0c6 P 279f8dbdfe22464a99e23b1b3b3d798b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:52.080332 18311 ts_tablet_manager.cc:1905] T d5f496d4092e487c82b9cc8b31b7f0c6 P 5e24f714afcb4bb8bec87f75aae157ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:07:52.081396 18310 ts_tablet_manager.cc:1905] T d5f496d4092e487c82b9cc8b31b7f0c6 P 279f8dbdfe22464a99e23b1b3b3d798b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:07:52.083326 18311 ts_tablet_manager.cc:1918] T d5f496d4092e487c82b9cc8b31b7f0c6 P 5e24f714afcb4bb8bec87f75aae157ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:07:52.083464 18311 log.cc:1196] T d5f496d4092e487c82b9cc8b31b7f0c6 P 5e24f714afcb4bb8bec87f75aae157ce: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-2/wal/wals/d5f496d4092e487c82b9cc8b31b7f0c6
I20260812 19:07:52.083804 18311 ts_tablet_manager.cc:1939] T d5f496d4092e487c82b9cc8b31b7f0c6 P 5e24f714afcb4bb8bec87f75aae157ce: Deleting consensus metadata
I20260812 19:07:52.084018 18310 ts_tablet_manager.cc:1918] T d5f496d4092e487c82b9cc8b31b7f0c6 P 279f8dbdfe22464a99e23b1b3b3d798b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:07:52.085006 18310 log.cc:1196] T d5f496d4092e487c82b9cc8b31b7f0c6 P 279f8dbdfe22464a99e23b1b3b3d798b: Deleting WAL directory at /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561415988404-26120-0/minicluster-data/ts-1/wal/wals/d5f496d4092e487c82b9cc8b31b7f0c6
I20260812 19:07:52.085296 18310 ts_tablet_manager.cc:1939] T d5f496d4092e487c82b9cc8b31b7f0c6 P 279f8dbdfe22464a99e23b1b3b3d798b: Deleting consensus metadata
I20260812 19:07:52.085315 17703 catalog_manager.cc:4841] TS 5e24f714afcb4bb8bec87f75aae157ce (127.25.130.3:39955): tablet d5f496d4092e487c82b9cc8b31b7f0c6 (table test-table [id=bc50c1d152764798b74af7b9c32d348e]) successfully deleted
I20260812 19:07:52.086380 17703 catalog_manager.cc:4841] TS 279f8dbdfe22464a99e23b1b3b3d798b (127.25.130.2:35835): tablet d5f496d4092e487c82b9cc8b31b7f0c6 (table test-table [id=bc50c1d152764798b74af7b9c32d348e]) successfully deleted
W20260812 19:07:52.090272 17702 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.130.1:34443 (error 108)
I20260812 19:07:52.090404 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 17950
I20260812 19:07:52.098160 18277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:52.097493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.1:34443 (local address 127.0.0.1:42134)
0812 19:07:52.097911 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0812 19:07:52.098025 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.1:34443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301}
W20260812 19:07:52.098562 17702 catalog_manager.cc:4584] TS 512c19c07beb423a96942f1b71589100 (127.25.130.1:34443): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d5f496d4092e487c82b9cc8b31b7f0c6: Network error: Client connection negotiation failed: client connection to 127.25.130.1:34443: connect: Connection refused (error 111)
I20260812 19:07:52.103099 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 18111
I20260812 19:07:52.112234 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 17684
2026-08-12T19:07:52Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1264 ms)
[----------] 6 tests from EncryptionPolicyTest (10850 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:07:52 dist-test-slave-49h3 krb5kdc[18318](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:07:52 dist-test-slave-49h3 krb5kdc[18318](info): set up 2 sockets
Aug 12 19:07:52 dist-test-slave-49h3 krb5kdc[18318](info): commencing operation
krb5kdc: starting...
W20260812 19:07:53.170266 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:07:53 dist-test-slave-49h3 krb5kdc[18318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:07:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:07:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 19:07:53.299863 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36073
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:07:53.413266 18335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:53.413606 18335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:53.413672 18335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:53.418108 18335 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:07:53.418203 18335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:53.418236 18335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:53.418256 18335 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:07:53.418277 18335 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:07:53.424034 18335 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36073
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:53.425191 18335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:53.426297 18335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:53.435137 18344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:53.435165 18340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:53.435231 18335 server_base.cc:1034] running on GCE node
W20260812 19:07:53.435138 18342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:53.436002 18335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:53.437143 18335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:53.438311 18335 hybrid_clock.cc:648] HybridClock initialized: now 1786561673438286 us; error 42 us; skew 500 ppm
Aug 12 19:07:53 dist-test-slave-49h3 krb5kdc[18318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561673, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:07:53.449717 18335 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 19:07:53.450809 18335 webserver.cc:460] Webserver started at http://127.0.0.1:37081/ using document root <none> and password file <none>
I20260812 19:07:53.451404 18335 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:53.451469 18335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:53.451705 18335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:53.454028 18335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "eccf4ae1574e492996377d0b20fbe1a6"
format_stamp: "Formatted at 2026-08-12 19:07:53 on dist-test-slave-49h3"
server_key: "43a0204cba0d30e2617a7c516f92c070"
server_key_iv: "3625e74c2701ea576e703382be007fba"
server_key_version: "encryptionkey@0"
I20260812 19:07:53.454634 18335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "eccf4ae1574e492996377d0b20fbe1a6"
format_stamp: "Formatted at 2026-08-12 19:07:53 on dist-test-slave-49h3"
server_key: "43a0204cba0d30e2617a7c516f92c070"
server_key_iv: "3625e74c2701ea576e703382be007fba"
server_key_version: "encryptionkey@0"
I20260812 19:07:53.458603 18335 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 19:07:53.461175 18352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:53.462473 18335 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:07:53.462610 18335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "eccf4ae1574e492996377d0b20fbe1a6"
format_stamp: "Formatted at 2026-08-12 19:07:53 on dist-test-slave-49h3"
server_key: "43a0204cba0d30e2617a7c516f92c070"
server_key_iv: "3625e74c2701ea576e703382be007fba"
server_key_version: "encryptionkey@0"
I20260812 19:07:53.462723 18335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:53.479467 18335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:53.482759 18335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:53.482959 18335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:53.490337 18335 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36073
I20260812 19:07:53.490366 18414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36073 every 8 connection(s)
I20260812 19:07:53.491394 18335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:07:53.494341 18415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:53.497509 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 18335
I20260812 19:07:53.497668 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:07:53.497993 26120 external_mini_cluster.cc:1428] Setting key 698a0a6690271ac84b50567b45b8ea5a
I20260812 19:07:53.500888 18415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eccf4ae1574e492996377d0b20fbe1a6: Bootstrap starting.
I20260812 19:07:53.503592 18415 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eccf4ae1574e492996377d0b20fbe1a6: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:53.504419 18415 log.cc:824] T 00000000000000000000000000000000 P eccf4ae1574e492996377d0b20fbe1a6: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:53.507197 18415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eccf4ae1574e492996377d0b20fbe1a6: No bootstrap required, opened a new log
Aug 12 19:07:53 dist-test-slave-49h3 krb5kdc[18318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 19:07:53.510890 18415 raft_consensus.cc:348] T 00000000000000000000000000000000 P eccf4ae1574e492996377d0b20fbe1a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eccf4ae1574e492996377d0b20fbe1a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36073 } }
I20260812 19:07:53.511149 18415 raft_consensus.cc:374] T 00000000000000000000000000000000 P eccf4ae1574e492996377d0b20fbe1a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:53.511211 18415 raft_consensus.cc:729] T 00000000000000000000000000000000 P eccf4ae1574e492996377d0b20fbe1a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eccf4ae1574e492996377d0b20fbe1a6, State: Initialized, Role: FOLLOWER
I20260812 19:07:53.511682 18415 consensus_queue.cc:260] T 00000000000000000000000000000000 P eccf4ae1574e492996377d0b20fbe1a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eccf4ae1574e492996377d0b20fbe1a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36073 } }
I20260812 19:07:53.511829 18415 raft_consensus.cc:388] T 00000000000000000000000000000000 P eccf4ae1574e492996377d0b20fbe1a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:07:53.511915 18415 raft_consensus.cc:482] T 00000000000000000000000000000000 P eccf4ae1574e492996377d0b20fbe1a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:07:53.512022 18415 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eccf4ae1574e492996377d0b20fbe1a6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:53.513046 18415 raft_consensus.cc:504] T 00000000000000000000000000000000 P eccf4ae1574e492996377d0b20fbe1a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eccf4ae1574e492996377d0b20fbe1a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36073 } }
I20260812 19:07:53.513137 18419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:53.499759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56435 (local address 127.0.0.1:36073)
0812 19:07:53.500226 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:53.500242 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:53.500281 (+ 39us) server_negotiation.cc:407] Connection header received
0812 19:07:53.500941 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:53.500962 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:53.501356 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:53.501740 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:53.502589 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:53.503705 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:53.504580 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:53.505209 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:53.505368 (+ 159us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:53.508170 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:53.508175 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:53.508190 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:07:53.508221 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:53.509952 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:53.510482 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:53.510488 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:53.510493 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:53.510622 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:53.510946 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:53.510949 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:53.510950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:53.511429 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:53.511663 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:53.511972 (+ 309us) server_negotiation.cc:299] Negotiation successful
0812 19:07:53.512353 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":141,"threads_started":1}
I20260812 19:07:53.513430 18415 leader_election.cc:302] T 00000000000000000000000000000000 P eccf4ae1574e492996377d0b20fbe1a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eccf4ae1574e492996377d0b20fbe1a6; no voters:
I20260812 19:07:53.513821 18415 leader_election.cc:288] T 00000000000000000000000000000000 P eccf4ae1574e492996377d0b20fbe1a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:07:53.513940 18421 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eccf4ae1574e492996377d0b20fbe1a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:07:53.514173 18421 raft_consensus.cc:686] T 00000000000000000000000000000000 P eccf4ae1574e492996377d0b20fbe1a6 [term 1 LEADER]: Becoming Leader. State: Replica: eccf4ae1574e492996377d0b20fbe1a6, State: Running, Role: LEADER
I20260812 19:07:53.514531 18421 consensus_queue.cc:237] T 00000000000000000000000000000000 P eccf4ae1574e492996377d0b20fbe1a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eccf4ae1574e492996377d0b20fbe1a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36073 } }
I20260812 19:07:53.515156 18415 sys_catalog.cc:564] T 00000000000000000000000000000000 P eccf4ae1574e492996377d0b20fbe1a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:07:53.516445 18422 sys_catalog.cc:455] T 00000000000000000000000000000000 P eccf4ae1574e492996377d0b20fbe1a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eccf4ae1574e492996377d0b20fbe1a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eccf4ae1574e492996377d0b20fbe1a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36073 } } }
I20260812 19:07:53.516575 18422 sys_catalog.cc:458] T 00000000000000000000000000000000 P eccf4ae1574e492996377d0b20fbe1a6 [sys.catalog]: This master's current role is: LEADER
I20260812 19:07:53.516819 26120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:07:53.517264 18423 sys_catalog.cc:455] T 00000000000000000000000000000000 P eccf4ae1574e492996377d0b20fbe1a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eccf4ae1574e492996377d0b20fbe1a6. Latest consensus state: current_term: 1 leader_uuid: "eccf4ae1574e492996377d0b20fbe1a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eccf4ae1574e492996377d0b20fbe1a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36073 } } }
I20260812 19:07:53.517356 18423 sys_catalog.cc:458] T 00000000000000000000000000000000 P eccf4ae1574e492996377d0b20fbe1a6 [sys.catalog]: This master's current role is: LEADER
I20260812 19:07:53.519133 18431 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:07:53.522478 18431 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:07:53.526895 18419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:53.519516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36641 (local address 127.0.0.1:36073)
0812 19:07:53.519642 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:53.519648 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:53.519663 (+ 15us) server_negotiation.cc:407] Connection header received
0812 19:07:53.519701 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:53.519704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:53.519746 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:53.519810 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:53.520510 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:53.521055 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:53.521915 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:53.522154 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:53.522243 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:53.523675 (+ 1432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:53.523678 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:53.523680 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:07:53.523707 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:53.525064 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:53.525568 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:53.525571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:53.525573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:53.525667 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:53.525993 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:53.525995 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:53.525996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:53.526201 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:53.526334 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:53.526626 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 19:07:53.526728 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 19:07:53.529498 18431 catalog_manager.cc:1347] Generated new cluster ID: 58ede40dfbfc4758930bd2ee2db9e292
I20260812 19:07:53.529646 18431 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:07:53.541273 18431 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:07:53.542641 18431 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:07:53.550058 18431 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eccf4ae1574e492996377d0b20fbe1a6: Generated new TSK 0
I20260812 19:07:53.550804 18431 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:07:53.557399 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 18335
W20260812 19:07:53.567766 18313 connection.cc:535] client connection to 127.0.0.1:36073 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36073): Connection reset by peer (error 104)
2026-08-12T19:07:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1444 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:07:53 dist-test-slave-49h3 krb5kdc[18450](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:07:53 dist-test-slave-49h3 krb5kdc[18450](info): set up 2 sockets
Aug 12 19:07:53 dist-test-slave-49h3 krb5kdc[18450](info): commencing operation
krb5kdc: starting...
W20260812 19:07:54.611536 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:07:54 dist-test-slave-49h3 krb5kdc[18450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:07:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:07:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 19:07:54.736068 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33039
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:07:54.847694 18465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:54.847993 18465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:54.848034 18465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:54.852175 18465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:07:54.852258 18465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:54.852277 18465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:54.852293 18465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:07:54.852306 18465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:07:54.858227 18465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33039
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:54.859359 18465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:54.860392 18465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:54.869019 18472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:54.869019 18475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:54.869042 18473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:54.869300 18465 server_base.cc:1034] running on GCE node
I20260812 19:07:54.869901 18465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:54.871028 18465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:54.872213 18465 hybrid_clock.cc:648] HybridClock initialized: now 1786561674872200 us; error 28 us; skew 500 ppm
Aug 12 19:07:54 dist-test-slave-49h3 krb5kdc[18450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561674, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:07:54.874977 18465 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 19:07:54.875926 18465 webserver.cc:460] Webserver started at http://127.0.0.1:45233/ using document root <none> and password file <none>
I20260812 19:07:54.876475 18465 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:54.876544 18465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:54.876744 18465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:54.878862 18465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "adf9ff03e22d4c86a24a06bff60a3923"
format_stamp: "Formatted at 2026-08-12 19:07:54 on dist-test-slave-49h3"
server_key: "f7e37b7267cc81fade8b151347d6ef8b"
server_key_iv: "702752c871390a7f61871ef6de04880e"
server_key_version: "encryptionkey@0"
I20260812 19:07:54.879446 18465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "adf9ff03e22d4c86a24a06bff60a3923"
format_stamp: "Formatted at 2026-08-12 19:07:54 on dist-test-slave-49h3"
server_key: "f7e37b7267cc81fade8b151347d6ef8b"
server_key_iv: "702752c871390a7f61871ef6de04880e"
server_key_version: "encryptionkey@0"
I20260812 19:07:54.883358 18465 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:07:54.885895 18485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:54.887176 18465 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:07:54.887305 18465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "adf9ff03e22d4c86a24a06bff60a3923"
format_stamp: "Formatted at 2026-08-12 19:07:54 on dist-test-slave-49h3"
server_key: "f7e37b7267cc81fade8b151347d6ef8b"
server_key_iv: "702752c871390a7f61871ef6de04880e"
server_key_version: "encryptionkey@0"
I20260812 19:07:54.887414 18465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:54.897943 18465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:54.900889 18465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:54.901084 18465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:54.908219 18465 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33039
I20260812 19:07:54.908236 18557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33039 every 8 connection(s)
I20260812 19:07:54.909204 18465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:07:54.912063 18558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:54.913035 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 18465
I20260812 19:07:54.913190 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:07:54.913546 26120 external_mini_cluster.cc:1428] Setting key ddc951584de6abd0f4a13f396dfcc5a1
I20260812 19:07:54.918457 18558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adf9ff03e22d4c86a24a06bff60a3923: Bootstrap starting.
I20260812 19:07:54.920745 18558 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P adf9ff03e22d4c86a24a06bff60a3923: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:54.921505 18558 log.cc:824] T 00000000000000000000000000000000 P adf9ff03e22d4c86a24a06bff60a3923: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:54.922533 18563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:54.915028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55601 (local address 127.0.0.1:33039)
0812 19:07:54.915514 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:54.915530 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:54.915569 (+ 39us) server_negotiation.cc:407] Connection header received
0812 19:07:54.916206 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:54.916223 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:54.916569 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:54.916876 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:54.917296 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:54.918302 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:54.919221 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:54.919678 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:54.919821 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:54.920687 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:54.920693 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:54.920705 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:07:54.921134 (+ 429us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:54.921194 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:07:54.921218 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:54.921330 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:54.921522 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 19:07:54.921667 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":145,"threads_started":1}
I20260812 19:07:54.923628 18558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adf9ff03e22d4c86a24a06bff60a3923: No bootstrap required, opened a new log
I20260812 19:07:54.925345 26120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:07:54.927518 18558 raft_consensus.cc:348] T 00000000000000000000000000000000 P adf9ff03e22d4c86a24a06bff60a3923 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adf9ff03e22d4c86a24a06bff60a3923" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33039 } }
I20260812 19:07:54.927843 18558 raft_consensus.cc:374] T 00000000000000000000000000000000 P adf9ff03e22d4c86a24a06bff60a3923 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:54.927912 18558 raft_consensus.cc:729] T 00000000000000000000000000000000 P adf9ff03e22d4c86a24a06bff60a3923 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adf9ff03e22d4c86a24a06bff60a3923, State: Initialized, Role: FOLLOWER
I20260812 19:07:54.928362 18558 consensus_queue.cc:260] T 00000000000000000000000000000000 P adf9ff03e22d4c86a24a06bff60a3923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adf9ff03e22d4c86a24a06bff60a3923" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33039 } }
I20260812 19:07:54.928495 18558 raft_consensus.cc:388] T 00000000000000000000000000000000 P adf9ff03e22d4c86a24a06bff60a3923 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:07:54.928558 18558 raft_consensus.cc:482] T 00000000000000000000000000000000 P adf9ff03e22d4c86a24a06bff60a3923 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:07:54.928663 18558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adf9ff03e22d4c86a24a06bff60a3923 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:54.929898 18558 raft_consensus.cc:504] T 00000000000000000000000000000000 P adf9ff03e22d4c86a24a06bff60a3923 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adf9ff03e22d4c86a24a06bff60a3923" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33039 } }
I20260812 19:07:54.930267 18558 leader_election.cc:302] T 00000000000000000000000000000000 P adf9ff03e22d4c86a24a06bff60a3923 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: adf9ff03e22d4c86a24a06bff60a3923; no voters:
I20260812 19:07:54.930712 18558 leader_election.cc:288] T 00000000000000000000000000000000 P adf9ff03e22d4c86a24a06bff60a3923 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:07:54.931207 18572 raft_consensus.cc:2781] T 00000000000000000000000000000000 P adf9ff03e22d4c86a24a06bff60a3923 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:07:54.931761 18563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:54.927282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43357 (local address 127.0.0.1:33039)
0812 19:07:54.927397 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:54.927401 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:54.927556 (+ 155us) server_negotiation.cc:407] Connection header received
0812 19:07:54.927680 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:54.927682 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:54.927721 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:54.927780 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:54.928091 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:54.928741 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:54.929813 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:54.930103 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:54.930194 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:54.931215 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:54.931219 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:54.931221 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:07:54.931235 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:54.931264 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:07:54.931277 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:54.931332 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:54.931552 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 19:07:54.931593 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 19:07:54.931612 18572 raft_consensus.cc:686] T 00000000000000000000000000000000 P adf9ff03e22d4c86a24a06bff60a3923 [term 1 LEADER]: Becoming Leader. State: Replica: adf9ff03e22d4c86a24a06bff60a3923, State: Running, Role: LEADER
I20260812 19:07:54.932168 18558 sys_catalog.cc:564] T 00000000000000000000000000000000 P adf9ff03e22d4c86a24a06bff60a3923 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:07:54.932193 18572 consensus_queue.cc:237] T 00000000000000000000000000000000 P adf9ff03e22d4c86a24a06bff60a3923 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adf9ff03e22d4c86a24a06bff60a3923" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33039 } }
I20260812 19:07:54.933607 18572 sys_catalog.cc:455] T 00000000000000000000000000000000 P adf9ff03e22d4c86a24a06bff60a3923 [sys.catalog]: SysCatalogTable state changed. Reason: New leader adf9ff03e22d4c86a24a06bff60a3923. Latest consensus state: current_term: 1 leader_uuid: "adf9ff03e22d4c86a24a06bff60a3923" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adf9ff03e22d4c86a24a06bff60a3923" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33039 } } }
I20260812 19:07:54.933795 18572 sys_catalog.cc:458] T 00000000000000000000000000000000 P adf9ff03e22d4c86a24a06bff60a3923 [sys.catalog]: This master's current role is: LEADER
I20260812 19:07:54.934037 18571 sys_catalog.cc:455] T 00000000000000000000000000000000 P adf9ff03e22d4c86a24a06bff60a3923 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "adf9ff03e22d4c86a24a06bff60a3923" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adf9ff03e22d4c86a24a06bff60a3923" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33039 } } }
I20260812 19:07:54.934167 18571 sys_catalog.cc:458] T 00000000000000000000000000000000 P adf9ff03e22d4c86a24a06bff60a3923 [sys.catalog]: This master's current role is: LEADER
W20260812 19:07:54.936916 18591 catalog_manager.cc:1558] T 00000000000000000000000000000000 P adf9ff03e22d4c86a24a06bff60a3923: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:07:54.937003 18591 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:07:54.937119 18589 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:07:54.938139 18589 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:07:54.943962 18589 catalog_manager.cc:1347] Generated new cluster ID: 8f920bdbf79a40b8bc4d2388176ebf80
I20260812 19:07:54.944053 18589 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:07:54.954893 18589 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:07:54.955878 18589 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:07:54.968111 18589 catalog_manager.cc:5868] T 00000000000000000000000000000000 P adf9ff03e22d4c86a24a06bff60a3923: Generated new TSK 0
I20260812 19:07:54.968878 18589 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:07:54.980494 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 18465
W20260812 19:07:54.991428 18447 connection.cc:535] client connection to 127.0.0.1:33039 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33039): Connection reset by peer (error 104)
2026-08-12T19:07:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1423 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:07:55 dist-test-slave-49h3 krb5kdc[18601](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:07:55 dist-test-slave-49h3 krb5kdc[18601](info): set up 2 sockets
Aug 12 19:07:55 dist-test-slave-49h3 krb5kdc[18601](info): commencing operation
krb5kdc: starting...
W20260812 19:07:56.054075 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:07:56 dist-test-slave-49h3 krb5kdc[18601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:07:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:07:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 19:07:56.180277 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46647
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:07:56.295378 18616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:56.295661 18616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:56.295725 18616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:56.299845 18616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:07:56.299921 18616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:56.299952 18616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:56.299973 18616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:07:56.299994 18616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:07:56.305624 18616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46647
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:56.306789 18616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:56.307840 18616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:56.316362 18624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:56.316362 18626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:56.316548 18616 server_base.cc:1034] running on GCE node
W20260812 19:07:56.316386 18623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:56.317201 18616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:56.318359 18616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:56.319558 18616 hybrid_clock.cc:648] HybridClock initialized: now 1786561676319523 us; error 38 us; skew 500 ppm
Aug 12 19:07:56 dist-test-slave-49h3 krb5kdc[18601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561676, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:07:56.322395 18616 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 19:07:56.323321 18616 webserver.cc:460] Webserver started at http://127.0.0.1:46817/ using document root <none> and password file <none>
I20260812 19:07:56.323879 18616 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:56.323947 18616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:56.324147 18616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:56.326347 18616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "b64526f3cb444c72904b6a13962c4807"
format_stamp: "Formatted at 2026-08-12 19:07:56 on dist-test-slave-49h3"
server_key: "e4ce503a5cf9f8910320011e90e5c9b4"
server_key_iv: "67405545f16bc999d6763ffc3a7f9f2f"
server_key_version: "encryptionkey@0"
I20260812 19:07:56.326941 18616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "b64526f3cb444c72904b6a13962c4807"
format_stamp: "Formatted at 2026-08-12 19:07:56 on dist-test-slave-49h3"
server_key: "e4ce503a5cf9f8910320011e90e5c9b4"
server_key_iv: "67405545f16bc999d6763ffc3a7f9f2f"
server_key_version: "encryptionkey@0"
I20260812 19:07:56.330780 18616 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:07:56.333282 18638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:56.334558 18616 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:07:56.334693 18616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "b64526f3cb444c72904b6a13962c4807"
format_stamp: "Formatted at 2026-08-12 19:07:56 on dist-test-slave-49h3"
server_key: "e4ce503a5cf9f8910320011e90e5c9b4"
server_key_iv: "67405545f16bc999d6763ffc3a7f9f2f"
server_key_version: "encryptionkey@0"
I20260812 19:07:56.334800 18616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:56.339670 18616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:56.343087 18616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:56.343299 18616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:56.350420 18616 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46647
I20260812 19:07:56.350446 18710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46647 every 8 connection(s)
I20260812 19:07:56.351445 18616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:07:56.354382 18711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:56.357674 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 18616
I20260812 19:07:56.357811 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:07:56.358076 26120 external_mini_cluster.cc:1428] Setting key cee47a1076d3d2bb290a2b34bacfe39e
I20260812 19:07:56.360875 18711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b64526f3cb444c72904b6a13962c4807: Bootstrap starting.
I20260812 19:07:56.363816 18711 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b64526f3cb444c72904b6a13962c4807: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:56.364514 18715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:56.359604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54295 (local address 127.0.0.1:46647)
0812 19:07:56.360110 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:56.360131 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:56.360176 (+ 45us) server_negotiation.cc:407] Connection header received
0812 19:07:56.361061 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:56.361086 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:56.361515 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:56.361891 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:56.362707 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:56.362713 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:56.362728 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:07:56.362741 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:56.362807 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:07:56.362833 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:56.362964 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:56.363348 (+ 384us) server_negotiation.cc:299] Negotiation successful
0812 19:07:56.363507 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":145,"threads_started":1}
I20260812 19:07:56.364748 18711 log.cc:824] T 00000000000000000000000000000000 P b64526f3cb444c72904b6a13962c4807: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:56.367197 18711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b64526f3cb444c72904b6a13962c4807: No bootstrap required, opened a new log
I20260812 19:07:56.367633 26120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:07:56.371587 18715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:56.370027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47655 (local address 127.0.0.1:46647)
0812 19:07:56.370150 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:56.370155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:56.370370 (+ 215us) server_negotiation.cc:407] Connection header received
0812 19:07:56.370460 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:56.370464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:56.370506 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:56.370603 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:56.371239 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:56.371243 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:56.371245 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:07:56.371256 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:56.371286 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:07:56.371297 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:56.371386 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:56.371432 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 19:07:56.371468 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 19:07:56.371786 18711 raft_consensus.cc:348] T 00000000000000000000000000000000 P b64526f3cb444c72904b6a13962c4807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b64526f3cb444c72904b6a13962c4807" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46647 } }
I20260812 19:07:56.372053 18711 raft_consensus.cc:374] T 00000000000000000000000000000000 P b64526f3cb444c72904b6a13962c4807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:56.372112 18711 raft_consensus.cc:729] T 00000000000000000000000000000000 P b64526f3cb444c72904b6a13962c4807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b64526f3cb444c72904b6a13962c4807, State: Initialized, Role: FOLLOWER
I20260812 19:07:56.372573 18711 consensus_queue.cc:260] T 00000000000000000000000000000000 P b64526f3cb444c72904b6a13962c4807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b64526f3cb444c72904b6a13962c4807" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46647 } }
I20260812 19:07:56.372713 18711 raft_consensus.cc:388] T 00000000000000000000000000000000 P b64526f3cb444c72904b6a13962c4807 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:07:56.372759 18711 raft_consensus.cc:482] T 00000000000000000000000000000000 P b64526f3cb444c72904b6a13962c4807 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:07:56.372831 18711 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b64526f3cb444c72904b6a13962c4807 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:56.373791 18711 raft_consensus.cc:504] T 00000000000000000000000000000000 P b64526f3cb444c72904b6a13962c4807 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b64526f3cb444c72904b6a13962c4807" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46647 } }
I20260812 19:07:56.374140 18711 leader_election.cc:302] T 00000000000000000000000000000000 P b64526f3cb444c72904b6a13962c4807 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b64526f3cb444c72904b6a13962c4807; no voters:
I20260812 19:07:56.374455 18711 leader_election.cc:288] T 00000000000000000000000000000000 P b64526f3cb444c72904b6a13962c4807 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:07:56.374536 18722 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b64526f3cb444c72904b6a13962c4807 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:07:56.374753 18722 raft_consensus.cc:686] T 00000000000000000000000000000000 P b64526f3cb444c72904b6a13962c4807 [term 1 LEADER]: Becoming Leader. State: Replica: b64526f3cb444c72904b6a13962c4807, State: Running, Role: LEADER
I20260812 19:07:56.375087 18722 consensus_queue.cc:237] T 00000000000000000000000000000000 P b64526f3cb444c72904b6a13962c4807 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b64526f3cb444c72904b6a13962c4807" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46647 } }
I20260812 19:07:56.375645 18711 sys_catalog.cc:564] T 00000000000000000000000000000000 P b64526f3cb444c72904b6a13962c4807 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:07:56.376696 18725 sys_catalog.cc:455] T 00000000000000000000000000000000 P b64526f3cb444c72904b6a13962c4807 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b64526f3cb444c72904b6a13962c4807. Latest consensus state: current_term: 1 leader_uuid: "b64526f3cb444c72904b6a13962c4807" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b64526f3cb444c72904b6a13962c4807" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46647 } } }
I20260812 19:07:56.376696 18724 sys_catalog.cc:455] T 00000000000000000000000000000000 P b64526f3cb444c72904b6a13962c4807 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b64526f3cb444c72904b6a13962c4807" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b64526f3cb444c72904b6a13962c4807" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46647 } } }
I20260812 19:07:56.376876 18725 sys_catalog.cc:458] T 00000000000000000000000000000000 P b64526f3cb444c72904b6a13962c4807 [sys.catalog]: This master's current role is: LEADER
I20260812 19:07:56.376890 18724 sys_catalog.cc:458] T 00000000000000000000000000000000 P b64526f3cb444c72904b6a13962c4807 [sys.catalog]: This master's current role is: LEADER
I20260812 19:07:56.377244 18734 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:07:56.380208 18734 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:07:56.386852 18734 catalog_manager.cc:1347] Generated new cluster ID: 2829053bf4524d90ac8aac63273bf74c
I20260812 19:07:56.386927 18734 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:07:56.399153 18734 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:07:56.399472 18734 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:07:56.408398 18734 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b64526f3cb444c72904b6a13962c4807: Generated new TSK 0
I20260812 19:07:56.409096 18734 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:07:56.413802 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 18616
2026-08-12T19:07:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1431 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:07:56 dist-test-slave-49h3 krb5kdc[18748](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:07:56 dist-test-slave-49h3 krb5kdc[18748](info): set up 2 sockets
Aug 12 19:07:56 dist-test-slave-49h3 krb5kdc[18748](info): commencing operation
krb5kdc: starting...
W20260812 19:07:57.478215 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:07:57 dist-test-slave-49h3 krb5kdc[18748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:07:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:07:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 19:07:57.605415 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45105
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:07:57.717186 18762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:57.717518 18762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:57.717592 18762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:57.721650 18762 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:07:57.721721 18762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:57.721747 18762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:57.721767 18762 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:07:57.721788 18762 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:07:57.727283 18762 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45105
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:57.728441 18762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:57.729552 18762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:57.737882 18772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:57.737877 18768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:57.738070 18762 server_base.cc:1034] running on GCE node
W20260812 19:07:57.737871 18770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:57.738782 18762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:57.739881 18762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:57.741039 18762 hybrid_clock.cc:648] HybridClock initialized: now 1786561677741019 us; error 31 us; skew 500 ppm
Aug 12 19:07:57 dist-test-slave-49h3 krb5kdc[18748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561677, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:07:57.743724 18762 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 19:07:57.744643 18762 webserver.cc:460] Webserver started at http://127.0.0.1:45637/ using document root <none> and password file <none>
I20260812 19:07:57.745193 18762 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:57.745254 18762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:57.745491 18762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:57.747628 18762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "0146337421474a419c933913c5c1ac06"
format_stamp: "Formatted at 2026-08-12 19:07:57 on dist-test-slave-49h3"
server_key: "ae42c6c643a282e06cf804247f500392"
server_key_iv: "20963afdbbeeb926d2988274c60e1c73"
server_key_version: "encryptionkey@0"
I20260812 19:07:57.748210 18762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "0146337421474a419c933913c5c1ac06"
format_stamp: "Formatted at 2026-08-12 19:07:57 on dist-test-slave-49h3"
server_key: "ae42c6c643a282e06cf804247f500392"
server_key_iv: "20963afdbbeeb926d2988274c60e1c73"
server_key_version: "encryptionkey@0"
I20260812 19:07:57.752223 18762 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 19:07:57.754689 18779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:57.755905 18762 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 19:07:57.756021 18762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "0146337421474a419c933913c5c1ac06"
format_stamp: "Formatted at 2026-08-12 19:07:57 on dist-test-slave-49h3"
server_key: "ae42c6c643a282e06cf804247f500392"
server_key_iv: "20963afdbbeeb926d2988274c60e1c73"
server_key_version: "encryptionkey@0"
I20260812 19:07:57.756141 18762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:57.769222 18762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:57.772030 18762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:57.772229 18762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:57.779515 18762 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45105
I20260812 19:07:57.779541 18844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45105 every 8 connection(s)
I20260812 19:07:57.780519 18762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:07:57.781865 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 18762
I20260812 19:07:57.782001 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:07:57.782300 26120 external_mini_cluster.cc:1428] Setting key 8468ecec6988a8ca46d22e0e557a29b8
I20260812 19:07:57.783901 18845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:57.790565 18845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0146337421474a419c933913c5c1ac06: Bootstrap starting.
Aug 12 19:07:57 dist-test-slave-49h3 krb5kdc[18748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 19:07:57.793115 18845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0146337421474a419c933913c5c1ac06: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:57.793923 18845 log.cc:824] T 00000000000000000000000000000000 P 0146337421474a419c933913c5c1ac06: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:57.795989 18845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0146337421474a419c933913c5c1ac06: No bootstrap required, opened a new log
I20260812 19:07:57.796546 18851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:57.783708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42285 (local address 127.0.0.1:45105)
0812 19:07:57.784136 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:57.784152 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:57.784190 (+ 38us) server_negotiation.cc:407] Connection header received
0812 19:07:57.784829 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:57.784845 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:57.785202 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:57.785521 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:57.786424 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:57.787416 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:57.788565 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:57.789021 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:57.789142 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:57.791907 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:57.791912 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:57.791926 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:07:57.791957 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:57.793664 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:57.794103 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:57.794108 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:57.794113 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:57.794232 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:57.794498 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:57.794501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:57.794503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:57.794930 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:57.795115 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:57.795417 (+ 302us) server_negotiation.cc:299] Negotiation successful
0812 19:07:57.795731 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":127,"threads_started":1}
I20260812 19:07:57.799525 26120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:07:57.800101 18845 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0146337421474a419c933913c5c1ac06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0146337421474a419c933913c5c1ac06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45105 } }
I20260812 19:07:57.800338 18845 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0146337421474a419c933913c5c1ac06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:57.800396 18845 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0146337421474a419c933913c5c1ac06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0146337421474a419c933913c5c1ac06, State: Initialized, Role: FOLLOWER
I20260812 19:07:57.800870 18845 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0146337421474a419c933913c5c1ac06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0146337421474a419c933913c5c1ac06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45105 } }
I20260812 19:07:57.801010 18845 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0146337421474a419c933913c5c1ac06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:07:57.801086 18845 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0146337421474a419c933913c5c1ac06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:07:57.801188 18845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0146337421474a419c933913c5c1ac06 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:57.802232 18845 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0146337421474a419c933913c5c1ac06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0146337421474a419c933913c5c1ac06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45105 } }
I20260812 19:07:57.802553 18845 leader_election.cc:302] T 00000000000000000000000000000000 P 0146337421474a419c933913c5c1ac06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0146337421474a419c933913c5c1ac06; no voters:
I20260812 19:07:57.802955 18845 leader_election.cc:288] T 00000000000000000000000000000000 P 0146337421474a419c933913c5c1ac06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:07:57.803045 18858 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0146337421474a419c933913c5c1ac06 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:07:57.803267 18858 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0146337421474a419c933913c5c1ac06 [term 1 LEADER]: Becoming Leader. State: Replica: 0146337421474a419c933913c5c1ac06, State: Running, Role: LEADER
I20260812 19:07:57.803617 18858 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0146337421474a419c933913c5c1ac06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0146337421474a419c933913c5c1ac06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45105 } }
I20260812 19:07:57.804278 18845 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0146337421474a419c933913c5c1ac06 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:07:57.805215 18860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0146337421474a419c933913c5c1ac06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0146337421474a419c933913c5c1ac06. Latest consensus state: current_term: 1 leader_uuid: "0146337421474a419c933913c5c1ac06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0146337421474a419c933913c5c1ac06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45105 } } }
I20260812 19:07:57.805215 18859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0146337421474a419c933913c5c1ac06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0146337421474a419c933913c5c1ac06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0146337421474a419c933913c5c1ac06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45105 } } }
I20260812 19:07:57.805366 18859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0146337421474a419c933913c5c1ac06 [sys.catalog]: This master's current role is: LEADER
I20260812 19:07:57.805366 18860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0146337421474a419c933913c5c1ac06 [sys.catalog]: This master's current role is: LEADER
W20260812 19:07:57.809500 18876 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0146337421474a419c933913c5c1ac06: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:07:57.809651 18876 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:07:57.809755 18868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:07:57.809988 18851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:57.801435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53433 (local address 127.0.0.1:45105)
0812 19:07:57.801577 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:57.801581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:57.801773 (+ 192us) server_negotiation.cc:407] Connection header received
0812 19:07:57.801881 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:57.801884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:57.801928 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:57.801988 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:57.802690 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:57.803464 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:57.804605 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:57.804937 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:57.806609 (+ 1672us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:57.806803 (+ 194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:57.806807 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:57.806809 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:07:57.806833 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:57.808262 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:57.808658 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:57.808661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:57.808663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:57.808739 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:07:57.808983 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:07:57.808987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:07:57.808988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:07:57.809172 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:57.809289 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:57.809561 (+ 272us) server_negotiation.cc:299] Negotiation successful
0812 19:07:57.809674 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 19:07:57.810817 18868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:07:57.816596 18868 catalog_manager.cc:1347] Generated new cluster ID: f788c820588e4039bd7277fb6d61ed5a
I20260812 19:07:57.816684 18868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:07:57.840346 18868 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:07:57.841324 18868 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:07:57.853874 18868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0146337421474a419c933913c5c1ac06: Generated new TSK 0
I20260812 19:07:57.854820 18868 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:07:57.860775 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 18762
2026-08-12T19:07:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1446 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:07:57 dist-test-slave-49h3 krb5kdc[18884](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:07:57 dist-test-slave-49h3 krb5kdc[18884](info): set up 2 sockets
Aug 12 19:07:57 dist-test-slave-49h3 krb5kdc[18884](info): commencing operation
krb5kdc: starting...
W20260812 19:07:58.912909 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:07:58 dist-test-slave-49h3 krb5kdc[18884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:07:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:07:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 19:07:59.038805 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44603
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44603
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:07:59.152297 18905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:07:59.152591 18905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:07:59.152634 18905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:07:59.156719 18905 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:07:59.156802 18905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:07:59.156821 18905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:07:59.156837 18905 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:07:59.156852 18905 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:07:59.162706 18905 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:07:59.163800 18905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:07:59.164803 18905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:07:59.173249 18910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:59.173337 18911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:07:59.173282 18913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:07:59.173296 18905 server_base.cc:1034] running on GCE node
I20260812 19:07:59.174052 18905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:07:59.175231 18905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:07:59.176420 18905 hybrid_clock.cc:648] HybridClock initialized: now 1786561679176394 us; error 42 us; skew 500 ppm
Aug 12 19:07:59 dist-test-slave-49h3 krb5kdc[18884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561679, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:07:59.185492 18905 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 19:07:59.186554 18905 webserver.cc:460] Webserver started at http://127.0.0.1:35095/ using document root <none> and password file <none>
I20260812 19:07:59.187155 18905 fs_manager.cc:359] Metadata directory not provided
I20260812 19:07:59.187233 18905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:07:59.187462 18905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:07:59.189754 18905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "a7fb2c9f2db44abf966e2c01b7edf37c"
format_stamp: "Formatted at 2026-08-12 19:07:59 on dist-test-slave-49h3"
server_key: "7e5e7d12dc148318a656af444bec2700"
server_key_iv: "69a9397adaab2dfe8f8a83eaa06befac"
server_key_version: "encryptionkey@0"
I20260812 19:07:59.190357 18905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "a7fb2c9f2db44abf966e2c01b7edf37c"
format_stamp: "Formatted at 2026-08-12 19:07:59 on dist-test-slave-49h3"
server_key: "7e5e7d12dc148318a656af444bec2700"
server_key_iv: "69a9397adaab2dfe8f8a83eaa06befac"
server_key_version: "encryptionkey@0"
I20260812 19:07:59.194404 18905 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:07:59.196931 18921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:07:59.198236 18905 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:07:59.198365 18905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "a7fb2c9f2db44abf966e2c01b7edf37c"
format_stamp: "Formatted at 2026-08-12 19:07:59 on dist-test-slave-49h3"
server_key: "7e5e7d12dc148318a656af444bec2700"
server_key_iv: "69a9397adaab2dfe8f8a83eaa06befac"
server_key_version: "encryptionkey@0"
I20260812 19:07:59.198472 18905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:07:59.216959 18905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:07:59.219888 18905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:07:59.220083 18905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:07:59.227428 18905 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44603
I20260812 19:07:59.227447 18983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44603 every 8 connection(s)
I20260812 19:07:59.228446 18905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:07:59.231308 18985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:59.236235 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 18905
I20260812 19:07:59.236392 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:07:59.236683 26120 external_mini_cluster.cc:1428] Setting key 54745738f63ea9328c7c856e61c60d2a
I20260812 19:07:59.237324 18985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7fb2c9f2db44abf966e2c01b7edf37c: Bootstrap starting.
I20260812 19:07:59.239791 18985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7fb2c9f2db44abf966e2c01b7edf37c: Neither blocks nor log segments found. Creating new log.
I20260812 19:07:59.240653 18985 log.cc:824] T 00000000000000000000000000000000 P a7fb2c9f2db44abf966e2c01b7edf37c: Log is configured to *not* fsync() on all Append() calls
I20260812 19:07:59.243016 18985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7fb2c9f2db44abf966e2c01b7edf37c: No bootstrap required, opened a new log
I20260812 19:07:59.246080 18988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:59.238128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37137 (local address 127.0.0.1:44603)
0812 19:07:59.238548 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:59.238565 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:59.238602 (+ 37us) server_negotiation.cc:407] Connection header received
0812 19:07:59.239270 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:59.239288 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:59.239645 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:59.239971 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:59.240331 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:59.241571 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:59.242759 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:59.243272 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:59.243389 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:59.244263 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:59.244269 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:59.244283 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:07:59.244709 (+ 426us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:59.244771 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:07:59.244795 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:59.244872 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:59.245018 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 19:07:59.245253 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":110,"threads_started":1}
I20260812 19:07:59.247066 18985 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7fb2c9f2db44abf966e2c01b7edf37c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7fb2c9f2db44abf966e2c01b7edf37c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44603 } }
I20260812 19:07:59.247298 18985 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7fb2c9f2db44abf966e2c01b7edf37c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:07:59.247347 18985 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7fb2c9f2db44abf966e2c01b7edf37c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7fb2c9f2db44abf966e2c01b7edf37c, State: Initialized, Role: FOLLOWER
I20260812 19:07:59.247763 18985 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7fb2c9f2db44abf966e2c01b7edf37c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7fb2c9f2db44abf966e2c01b7edf37c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44603 } }
I20260812 19:07:59.247897 18985 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7fb2c9f2db44abf966e2c01b7edf37c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:07:59.247965 18985 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7fb2c9f2db44abf966e2c01b7edf37c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:07:59.248063 18985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7fb2c9f2db44abf966e2c01b7edf37c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:07:59.248872 26120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:07:59.249083 18985 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7fb2c9f2db44abf966e2c01b7edf37c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7fb2c9f2db44abf966e2c01b7edf37c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44603 } }
I20260812 19:07:59.249478 18985 leader_election.cc:302] T 00000000000000000000000000000000 P a7fb2c9f2db44abf966e2c01b7edf37c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7fb2c9f2db44abf966e2c01b7edf37c; no voters:
I20260812 19:07:59.249857 18985 leader_election.cc:288] T 00000000000000000000000000000000 P a7fb2c9f2db44abf966e2c01b7edf37c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:07:59.249984 18990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7fb2c9f2db44abf966e2c01b7edf37c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:07:59.250229 18990 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7fb2c9f2db44abf966e2c01b7edf37c [term 1 LEADER]: Becoming Leader. State: Replica: a7fb2c9f2db44abf966e2c01b7edf37c, State: Running, Role: LEADER
I20260812 19:07:59.250617 18990 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7fb2c9f2db44abf966e2c01b7edf37c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7fb2c9f2db44abf966e2c01b7edf37c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44603 } }
I20260812 19:07:59.251250 18985 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7fb2c9f2db44abf966e2c01b7edf37c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:07:59.251765 18992 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7fb2c9f2db44abf966e2c01b7edf37c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7fb2c9f2db44abf966e2c01b7edf37c. Latest consensus state: current_term: 1 leader_uuid: "a7fb2c9f2db44abf966e2c01b7edf37c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7fb2c9f2db44abf966e2c01b7edf37c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44603 } } }
I20260812 19:07:59.251958 18992 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7fb2c9f2db44abf966e2c01b7edf37c [sys.catalog]: This master's current role is: LEADER
I20260812 19:07:59.252162 18994 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7fb2c9f2db44abf966e2c01b7edf37c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7fb2c9f2db44abf966e2c01b7edf37c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7fb2c9f2db44abf966e2c01b7edf37c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44603 } } }
I20260812 19:07:59.252283 18994 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7fb2c9f2db44abf966e2c01b7edf37c [sys.catalog]: This master's current role is: LEADER
I20260812 19:07:59.252498 19003 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:07:59.255934 19003 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:07:59.256621 18988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:07:59.251458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45989 (local address 127.0.0.1:44603)
0812 19:07:59.251570 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 19:07:59.251575 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:07:59.251871 (+ 296us) server_negotiation.cc:407] Connection header received
0812 19:07:59.252054 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:07:59.252057 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:07:59.252110 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:07:59.252164 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:07:59.252624 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:59.253379 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:59.254514 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:07:59.254853 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:07:59.254969 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:07:59.255863 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:07:59.255868 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:07:59.255870 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:07:59.255885 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:07:59.255915 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:07:59.255930 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:07:59.256008 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 19:07:59.256422 (+ 414us) server_negotiation.cc:299] Negotiation successful
0812 19:07:59.256483 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 19:07:59.262068 19003 catalog_manager.cc:1347] Generated new cluster ID: f451e718ffc34ca7929d9b395158cb15
I20260812 19:07:59.262149 19003 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:07:59.286289 19003 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:07:59.287295 19003 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:07:59.293751 19003 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7fb2c9f2db44abf966e2c01b7edf37c: Generated new TSK 0
I20260812 19:07:59.294427 19003 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:07:59.306767 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 18905
2026-08-12T19:07:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1445 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:07:59 dist-test-slave-49h3 krb5kdc[19021](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:07:59 dist-test-slave-49h3 krb5kdc[19021](info): set up 2 sockets
Aug 12 19:07:59 dist-test-slave-49h3 krb5kdc[19021](info): commencing operation
krb5kdc: starting...
W20260812 19:08:00.358112 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:08:00 dist-test-slave-49h3 krb5kdc[19021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:08:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:08:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 19:08:00.486191 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36025
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:08:00.602402 19035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:08:00.602708 19035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:08:00.602771 19035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:08:00.606968 19035 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:08:00.607060 19035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:08:00.607088 19035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:08:00.607112 19035 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:08:00.607141 19035 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:08:00.612864 19035 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36025
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:08:00.614177 19035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:08:00.615247 19035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:08:00.623993 19041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:08:00.623993 19044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:08:00.623998 19042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:00.624114 19035 server_base.cc:1034] running on GCE node
I20260812 19:08:00.624817 19035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:08:00.626056 19035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:08:00.627223 19035 hybrid_clock.cc:648] HybridClock initialized: now 1786561680627203 us; error 42 us; skew 500 ppm
Aug 12 19:08:00 dist-test-slave-49h3 krb5kdc[19021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561680, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:08:00.630028 19035 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 19:08:00.630988 19035 webserver.cc:460] Webserver started at http://127.0.0.1:41123/ using document root <none> and password file <none>
I20260812 19:08:00.631551 19035 fs_manager.cc:359] Metadata directory not provided
I20260812 19:08:00.631621 19035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:08:00.631832 19035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:08:00.634092 19035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "b895137333a64a2a8b7f014feeeaa9e7"
format_stamp: "Formatted at 2026-08-12 19:08:00 on dist-test-slave-49h3"
server_key: "a690ceab8f2bfda5d8ddb93111ac80c4"
server_key_iv: "3e8bb50ecc111dcafc25678680ed9943"
server_key_version: "encryptionkey@0"
I20260812 19:08:00.634668 19035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "b895137333a64a2a8b7f014feeeaa9e7"
format_stamp: "Formatted at 2026-08-12 19:08:00 on dist-test-slave-49h3"
server_key: "a690ceab8f2bfda5d8ddb93111ac80c4"
server_key_iv: "3e8bb50ecc111dcafc25678680ed9943"
server_key_version: "encryptionkey@0"
I20260812 19:08:00.638638 19035 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:08:00.641168 19051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:08:00.642575 19035 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:08:00.642721 19035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "b895137333a64a2a8b7f014feeeaa9e7"
format_stamp: "Formatted at 2026-08-12 19:08:00 on dist-test-slave-49h3"
server_key: "a690ceab8f2bfda5d8ddb93111ac80c4"
server_key_iv: "3e8bb50ecc111dcafc25678680ed9943"
server_key_version: "encryptionkey@0"
I20260812 19:08:00.642843 19035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:08:00.648206 19035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:08:00.651427 19035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:08:00.651640 19035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:08:00.659015 19035 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36025
I20260812 19:08:00.659029 19116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36025 every 8 connection(s)
I20260812 19:08:00.660040 19035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:08:00.662976 19117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:08:00.663331 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 19035
I20260812 19:08:00.663453 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:08:00.663735 26120 external_mini_cluster.cc:1428] Setting key 8cbae481a501d78ff2f7931b3b86aaee
I20260812 19:08:00.669121 19120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:00.665156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56269 (local address 127.0.0.1:36025)
0812 19:08:00.665657 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:00.665673 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:00.665712 (+ 39us) server_negotiation.cc:407] Connection header received
0812 19:08:00.666408 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:00.666427 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:00.666731 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:00.667030 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:00.667902 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:00.667907 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:00.667919 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:08:00.667930 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:00.667986 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:08:00.668004 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:00.668119 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:00.668211 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 19:08:00.668358 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":153,"threads_started":1}
I20260812 19:08:00.670033 19117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b895137333a64a2a8b7f014feeeaa9e7: Bootstrap starting.
I20260812 19:08:00.671880 26120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:08:00.672519 19117 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b895137333a64a2a8b7f014feeeaa9e7: Neither blocks nor log segments found. Creating new log.
I20260812 19:08:00.673516 19117 log.cc:824] T 00000000000000000000000000000000 P b895137333a64a2a8b7f014feeeaa9e7: Log is configured to *not* fsync() on all Append() calls
I20260812 19:08:00.676030 19117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b895137333a64a2a8b7f014feeeaa9e7: No bootstrap required, opened a new log
I20260812 19:08:00.676007 19120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:00.674319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40679 (local address 127.0.0.1:36025)
0812 19:08:00.674430 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:00.674434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:00.674691 (+ 257us) server_negotiation.cc:407] Connection header received
0812 19:08:00.674855 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:00.674858 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:00.674894 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:00.674967 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:00.675652 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:00.675655 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:00.675657 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:08:00.675666 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:00.675689 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:08:00.675698 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:00.675759 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:00.675855 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 19:08:00.675889 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 19:08:00.679821 19117 raft_consensus.cc:348] T 00000000000000000000000000000000 P b895137333a64a2a8b7f014feeeaa9e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b895137333a64a2a8b7f014feeeaa9e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36025 } }
I20260812 19:08:00.680104 19117 raft_consensus.cc:374] T 00000000000000000000000000000000 P b895137333a64a2a8b7f014feeeaa9e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:08:00.680158 19117 raft_consensus.cc:729] T 00000000000000000000000000000000 P b895137333a64a2a8b7f014feeeaa9e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b895137333a64a2a8b7f014feeeaa9e7, State: Initialized, Role: FOLLOWER
I20260812 19:08:00.680605 19117 consensus_queue.cc:260] T 00000000000000000000000000000000 P b895137333a64a2a8b7f014feeeaa9e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b895137333a64a2a8b7f014feeeaa9e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36025 } }
I20260812 19:08:00.680742 19117 raft_consensus.cc:388] T 00000000000000000000000000000000 P b895137333a64a2a8b7f014feeeaa9e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:08:00.680812 19117 raft_consensus.cc:482] T 00000000000000000000000000000000 P b895137333a64a2a8b7f014feeeaa9e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:08:00.680914 19117 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b895137333a64a2a8b7f014feeeaa9e7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:08:00.682004 19117 raft_consensus.cc:504] T 00000000000000000000000000000000 P b895137333a64a2a8b7f014feeeaa9e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b895137333a64a2a8b7f014feeeaa9e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36025 } }
I20260812 19:08:00.682367 19117 leader_election.cc:302] T 00000000000000000000000000000000 P b895137333a64a2a8b7f014feeeaa9e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b895137333a64a2a8b7f014feeeaa9e7; no voters:
I20260812 19:08:00.682688 19117 leader_election.cc:288] T 00000000000000000000000000000000 P b895137333a64a2a8b7f014feeeaa9e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:08:00.682798 19128 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b895137333a64a2a8b7f014feeeaa9e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:08:00.683039 19128 raft_consensus.cc:686] T 00000000000000000000000000000000 P b895137333a64a2a8b7f014feeeaa9e7 [term 1 LEADER]: Becoming Leader. State: Replica: b895137333a64a2a8b7f014feeeaa9e7, State: Running, Role: LEADER
I20260812 19:08:00.683418 19128 consensus_queue.cc:237] T 00000000000000000000000000000000 P b895137333a64a2a8b7f014feeeaa9e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b895137333a64a2a8b7f014feeeaa9e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36025 } }
I20260812 19:08:00.683954 19117 sys_catalog.cc:564] T 00000000000000000000000000000000 P b895137333a64a2a8b7f014feeeaa9e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:08:00.685146 19129 sys_catalog.cc:455] T 00000000000000000000000000000000 P b895137333a64a2a8b7f014feeeaa9e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b895137333a64a2a8b7f014feeeaa9e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b895137333a64a2a8b7f014feeeaa9e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36025 } } }
I20260812 19:08:00.685146 19130 sys_catalog.cc:455] T 00000000000000000000000000000000 P b895137333a64a2a8b7f014feeeaa9e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b895137333a64a2a8b7f014feeeaa9e7. Latest consensus state: current_term: 1 leader_uuid: "b895137333a64a2a8b7f014feeeaa9e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b895137333a64a2a8b7f014feeeaa9e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36025 } } }
I20260812 19:08:00.685320 19129 sys_catalog.cc:458] T 00000000000000000000000000000000 P b895137333a64a2a8b7f014feeeaa9e7 [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:00.685422 19130 sys_catalog.cc:458] T 00000000000000000000000000000000 P b895137333a64a2a8b7f014feeeaa9e7 [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:00.685860 19142 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:08:00.688787 19142 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:08:00.694108 19142 catalog_manager.cc:1347] Generated new cluster ID: 3e554cc9d411428399776ecdacc6fb2a
I20260812 19:08:00.694190 19142 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:08:00.711009 19142 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:08:00.711390 19142 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:08:00.719910 19142 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b895137333a64a2a8b7f014feeeaa9e7: Generated new TSK 0
I20260812 19:08:00.720808 19142 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:08:00.730134 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 19035
2026-08-12T19:08:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1423 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:08:00 dist-test-slave-49h3 krb5kdc[19156](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:08:00 dist-test-slave-49h3 krb5kdc[19156](info): set up 2 sockets
Aug 12 19:08:00 dist-test-slave-49h3 krb5kdc[19156](info): commencing operation
krb5kdc: starting...
W20260812 19:08:01.781781 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:08:01 dist-test-slave-49h3 krb5kdc[19156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:08:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:08:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 19:08:01.906675 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42023
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42023
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:08:02.020897 19172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:08:02.021215 19172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:08:02.021281 19172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:08:02.025429 19172 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:08:02.025534 19172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:08:02.025563 19172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:08:02.025584 19172 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:08:02.025606 19172 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:08:02.031181 19172 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42023
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:08:02.032316 19172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:08:02.033388 19172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:08:02.043226 19182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:08:02.043226 19179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:08:02.043288 19180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:02.044395 19172 server_base.cc:1034] running on GCE node
I20260812 19:08:02.045254 19172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:08:02.046895 19172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:08:02.048069 19172 hybrid_clock.cc:648] HybridClock initialized: now 1786561682048040 us; error 45 us; skew 500 ppm
Aug 12 19:08:02 dist-test-slave-49h3 krb5kdc[19156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561682, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:08:02.051584 19172 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 19:08:02.053030 19172 webserver.cc:460] Webserver started at http://127.0.0.1:35245/ using document root <none> and password file <none>
I20260812 19:08:02.053896 19172 fs_manager.cc:359] Metadata directory not provided
I20260812 19:08:02.053992 19172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:08:02.054342 19172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:08:02.056922 19172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "c69c4f8e77e449bcb8871850816f8d11"
format_stamp: "Formatted at 2026-08-12 19:08:02 on dist-test-slave-49h3"
server_key: "a3db6d0dc06b7bae9b1096fedafe7cba"
server_key_iv: "0c481a2c46c03b8ec26a795d21dab6a0"
server_key_version: "encryptionkey@0"
I20260812 19:08:02.057619 19172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "c69c4f8e77e449bcb8871850816f8d11"
format_stamp: "Formatted at 2026-08-12 19:08:02 on dist-test-slave-49h3"
server_key: "a3db6d0dc06b7bae9b1096fedafe7cba"
server_key_iv: "0c481a2c46c03b8ec26a795d21dab6a0"
server_key_version: "encryptionkey@0"
I20260812 19:08:02.062260 19172 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 19:08:02.065412 19189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:08:02.067013 19172 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 19:08:02.067152 19172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "c69c4f8e77e449bcb8871850816f8d11"
format_stamp: "Formatted at 2026-08-12 19:08:02 on dist-test-slave-49h3"
server_key: "a3db6d0dc06b7bae9b1096fedafe7cba"
server_key_iv: "0c481a2c46c03b8ec26a795d21dab6a0"
server_key_version: "encryptionkey@0"
I20260812 19:08:02.067276 19172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:08:02.092236 19172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:08:02.096287 19172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:08:02.096552 19172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:08:02.105195 19172 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42023
I20260812 19:08:02.105229 19253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42023 every 8 connection(s)
I20260812 19:08:02.106374 19172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:08:02.109843 19254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:08:02.114225 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 19172
I20260812 19:08:02.114410 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:08:02.114816 26120 external_mini_cluster.cc:1428] Setting key 89f14727ea415184b13abcd4f0d45690
I20260812 19:08:02.117187 19254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c69c4f8e77e449bcb8871850816f8d11: Bootstrap starting.
I20260812 19:08:02.120338 19254 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c69c4f8e77e449bcb8871850816f8d11: Neither blocks nor log segments found. Creating new log.
I20260812 19:08:02.121258 19254 log.cc:824] T 00000000000000000000000000000000 P c69c4f8e77e449bcb8871850816f8d11: Log is configured to *not* fsync() on all Append() calls
I20260812 19:08:02.123910 19254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c69c4f8e77e449bcb8871850816f8d11: No bootstrap required, opened a new log
Aug 12 19:08:02 dist-test-slave-49h3 krb5kdc[19156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 19:08:02.127880 19254 raft_consensus.cc:348] T 00000000000000000000000000000000 P c69c4f8e77e449bcb8871850816f8d11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69c4f8e77e449bcb8871850816f8d11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42023 } }
I20260812 19:08:02.128168 19254 raft_consensus.cc:374] T 00000000000000000000000000000000 P c69c4f8e77e449bcb8871850816f8d11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:08:02.128235 19254 raft_consensus.cc:729] T 00000000000000000000000000000000 P c69c4f8e77e449bcb8871850816f8d11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c69c4f8e77e449bcb8871850816f8d11, State: Initialized, Role: FOLLOWER
I20260812 19:08:02.128777 19254 consensus_queue.cc:260] T 00000000000000000000000000000000 P c69c4f8e77e449bcb8871850816f8d11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69c4f8e77e449bcb8871850816f8d11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42023 } }
I20260812 19:08:02.128942 19254 raft_consensus.cc:388] T 00000000000000000000000000000000 P c69c4f8e77e449bcb8871850816f8d11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:08:02.129016 19254 raft_consensus.cc:482] T 00000000000000000000000000000000 P c69c4f8e77e449bcb8871850816f8d11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:08:02.129132 19254 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c69c4f8e77e449bcb8871850816f8d11 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:08:02.130270 19254 raft_consensus.cc:504] T 00000000000000000000000000000000 P c69c4f8e77e449bcb8871850816f8d11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69c4f8e77e449bcb8871850816f8d11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42023 } }
I20260812 19:08:02.130656 19254 leader_election.cc:302] T 00000000000000000000000000000000 P c69c4f8e77e449bcb8871850816f8d11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c69c4f8e77e449bcb8871850816f8d11; no voters:
I20260812 19:08:02.131095 19254 leader_election.cc:288] T 00000000000000000000000000000000 P c69c4f8e77e449bcb8871850816f8d11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:08:02.131206 19261 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c69c4f8e77e449bcb8871850816f8d11 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:08:02.131480 19261 raft_consensus.cc:686] T 00000000000000000000000000000000 P c69c4f8e77e449bcb8871850816f8d11 [term 1 LEADER]: Becoming Leader. State: Replica: c69c4f8e77e449bcb8871850816f8d11, State: Running, Role: LEADER
I20260812 19:08:02.131899 19261 consensus_queue.cc:237] T 00000000000000000000000000000000 P c69c4f8e77e449bcb8871850816f8d11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69c4f8e77e449bcb8871850816f8d11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42023 } }
I20260812 19:08:02.132718 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:02.116857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:35073 (local address 127.0.0.1:42023)
0812 19:08:02.117361 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:02.117403 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:02.117478 (+ 75us) server_negotiation.cc:407] Connection header received
0812 19:08:02.118239 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:02.118258 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:02.118688 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:02.119027 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:02.120130 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:02.121325 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:02.122486 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:02.123015 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:02.123146 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:02.126513 (+ 3367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:02.126519 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:02.126536 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:02.126570 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:02.128960 (+ 2390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:02.129577 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:02.129582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:02.129587 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:02.129712 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:02.130060 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:02.130063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:02.130064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:02.130568 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:02.130791 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:02.131163 (+ 372us) server_negotiation.cc:299] Negotiation successful
0812 19:08:02.131640 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":158,"threads_started":1}
I20260812 19:08:02.132799 19254 sys_catalog.cc:564] T 00000000000000000000000000000000 P c69c4f8e77e449bcb8871850816f8d11 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:08:02.133875 19262 sys_catalog.cc:455] T 00000000000000000000000000000000 P c69c4f8e77e449bcb8871850816f8d11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c69c4f8e77e449bcb8871850816f8d11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69c4f8e77e449bcb8871850816f8d11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42023 } } }
I20260812 19:08:02.134052 19262 sys_catalog.cc:458] T 00000000000000000000000000000000 P c69c4f8e77e449bcb8871850816f8d11 [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:02.134346 19262 sys_catalog.cc:455] T 00000000000000000000000000000000 P c69c4f8e77e449bcb8871850816f8d11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c69c4f8e77e449bcb8871850816f8d11. Latest consensus state: current_term: 1 leader_uuid: "c69c4f8e77e449bcb8871850816f8d11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69c4f8e77e449bcb8871850816f8d11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42023 } } }
I20260812 19:08:02.134454 19262 sys_catalog.cc:458] T 00000000000000000000000000000000 P c69c4f8e77e449bcb8871850816f8d11 [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:02.134793 19270 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:08:02.136730 26120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:08:02.138902 19270 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:08:02.146306 19270 catalog_manager.cc:1347] Generated new cluster ID: 4c79ba8a9fd846a39ed145abc1ef7e84
I20260812 19:08:02.146396 19270 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:08:02.155490 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:02.142061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:53197 (local address 127.0.0.1:42023)
0812 19:08:02.142189 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:02.142194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:02.142526 (+ 332us) server_negotiation.cc:407] Connection header received
0812 19:08:02.142662 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:02.142665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:02.142724 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:02.142808 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:02.144163 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:02.145286 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:02.149871 (+ 4585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:02.150137 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:02.150236 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:02.151805 (+ 1569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:02.151809 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:02.151811 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:02.151839 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:02.153398 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:02.154009 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:02.154012 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:02.154014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:02.154112 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:02.154453 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:02.154456 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:02.154457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:02.154671 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:02.154813 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:02.155132 (+ 319us) server_negotiation.cc:299] Negotiation successful
0812 19:08:02.155245 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":48}
I20260812 19:08:02.159085 19270 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:08:02.160527 19270 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:08:02.165998 19270 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c69c4f8e77e449bcb8871850816f8d11: Generated new TSK 0
I20260812 19:08:02.167073 19270 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:08:02.178159 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 19172
W20260812 19:08:02.189785 19154 connection.cc:535] client connection to 127.0.0.1:42023 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42023): Connection reset by peer (error 104)
2026-08-12T19:08:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1451 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:08:02 dist-test-slave-49h3 krb5kdc[19290](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:08:02 dist-test-slave-49h3 krb5kdc[19290](info): set up 2 sockets
Aug 12 19:08:02 dist-test-slave-49h3 krb5kdc[19290](info): commencing operation
krb5kdc: starting...
W20260812 19:08:03.234596 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:08:03 dist-test-slave-49h3 krb5kdc[19290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:08:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:08:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 19:08:03.360000 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43183
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43183
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:08:03.477918 19305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:08:03.478221 19305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:08:03.478283 19305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:08:03.482332 19305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:08:03.482409 19305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:08:03.482435 19305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:08:03.482456 19305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:08:03.482477 19305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:08:03.487962 19305 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:08:03.489130 19305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:08:03.490209 19305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:08:03.498764 19305 server_base.cc:1034] running on GCE node
W20260812 19:08:03.498734 19313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:08:03.498710 19311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:08:03.498710 19310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:03.499583 19305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:08:03.500790 19305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:08:03.501950 19305 hybrid_clock.cc:648] HybridClock initialized: now 1786561683501918 us; error 44 us; skew 500 ppm
Aug 12 19:08:03 dist-test-slave-49h3 krb5kdc[19290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561683, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:08:03.504738 19305 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 19:08:03.505674 19305 webserver.cc:460] Webserver started at http://127.0.0.1:43751/ using document root <none> and password file <none>
I20260812 19:08:03.506250 19305 fs_manager.cc:359] Metadata directory not provided
I20260812 19:08:03.506312 19305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:08:03.506485 19305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:08:03.508610 19305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "98477a7a3389496189be5e82299bd4ca"
format_stamp: "Formatted at 2026-08-12 19:08:03 on dist-test-slave-49h3"
server_key: "b8b716af4452a07ce5e13a47ad1597c1"
server_key_iv: "b9b5de246fd3282c5326480486eab654"
server_key_version: "encryptionkey@0"
I20260812 19:08:03.509204 19305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "98477a7a3389496189be5e82299bd4ca"
format_stamp: "Formatted at 2026-08-12 19:08:03 on dist-test-slave-49h3"
server_key: "b8b716af4452a07ce5e13a47ad1597c1"
server_key_iv: "b9b5de246fd3282c5326480486eab654"
server_key_version: "encryptionkey@0"
I20260812 19:08:03.513283 19305 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 19:08:03.515936 19322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:08:03.517292 19305 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 19:08:03.517412 19305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "98477a7a3389496189be5e82299bd4ca"
format_stamp: "Formatted at 2026-08-12 19:08:03 on dist-test-slave-49h3"
server_key: "b8b716af4452a07ce5e13a47ad1597c1"
server_key_iv: "b9b5de246fd3282c5326480486eab654"
server_key_version: "encryptionkey@0"
I20260812 19:08:03.517560 19305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:08:03.540866 19305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:08:03.544103 19305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:08:03.544313 19305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:08:03.552069 19305 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43183
I20260812 19:08:03.552088 19385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43183 every 8 connection(s)
I20260812 19:08:03.553112 19305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:08:03.556291 19388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:08:03.557606 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 19305
I20260812 19:08:03.557744 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:08:03.558036 26120 external_mini_cluster.cc:1428] Setting key 929d3c856e788a56cfcb106d873fbdeb
I20260812 19:08:03.563777 19388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98477a7a3389496189be5e82299bd4ca: Bootstrap starting.
I20260812 19:08:03.566708 19388 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98477a7a3389496189be5e82299bd4ca: Neither blocks nor log segments found. Creating new log.
I20260812 19:08:03.567746 19388 log.cc:824] T 00000000000000000000000000000000 P 98477a7a3389496189be5e82299bd4ca: Log is configured to *not* fsync() on all Append() calls
I20260812 19:08:03.568817 19391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:03.559596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:53939 (local address 127.0.0.1:43183)
0812 19:08:03.560102 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:03.560123 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:03.560171 (+ 48us) server_negotiation.cc:407] Connection header received
0812 19:08:03.561044 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:03.561070 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:03.561599 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:03.562009 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:03.562485 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:03.563972 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:03.564906 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:03.565663 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:03.565824 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:03.566680 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:03.566687 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:03.566715 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:08:03.567257 (+ 542us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:03.567334 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:08:03.567367 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:03.567506 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:03.567662 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 19:08:03.567826 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":134,"threads_started":1}
I20260812 19:08:03.570102 19388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98477a7a3389496189be5e82299bd4ca: No bootstrap required, opened a new log
I20260812 19:08:03.571992 26120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:08:03.574291 19388 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98477a7a3389496189be5e82299bd4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98477a7a3389496189be5e82299bd4ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43183 } }
I20260812 19:08:03.574587 19388 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98477a7a3389496189be5e82299bd4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:08:03.574651 19388 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98477a7a3389496189be5e82299bd4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98477a7a3389496189be5e82299bd4ca, State: Initialized, Role: FOLLOWER
I20260812 19:08:03.575091 19388 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98477a7a3389496189be5e82299bd4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98477a7a3389496189be5e82299bd4ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43183 } }
I20260812 19:08:03.575240 19388 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98477a7a3389496189be5e82299bd4ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:08:03.575313 19388 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98477a7a3389496189be5e82299bd4ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:08:03.575417 19388 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98477a7a3389496189be5e82299bd4ca [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:08:03.576375 19388 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98477a7a3389496189be5e82299bd4ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98477a7a3389496189be5e82299bd4ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43183 } }
I20260812 19:08:03.576761 19388 leader_election.cc:302] T 00000000000000000000000000000000 P 98477a7a3389496189be5e82299bd4ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98477a7a3389496189be5e82299bd4ca; no voters:
I20260812 19:08:03.577128 19388 leader_election.cc:288] T 00000000000000000000000000000000 P 98477a7a3389496189be5e82299bd4ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:08:03.577241 19399 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98477a7a3389496189be5e82299bd4ca [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:08:03.577481 19399 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98477a7a3389496189be5e82299bd4ca [term 1 LEADER]: Becoming Leader. State: Replica: 98477a7a3389496189be5e82299bd4ca, State: Running, Role: LEADER
I20260812 19:08:03.577903 19399 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98477a7a3389496189be5e82299bd4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98477a7a3389496189be5e82299bd4ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43183 } }
I20260812 19:08:03.578605 19388 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98477a7a3389496189be5e82299bd4ca [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:08:03.578656 19391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:03.574209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:54697 (local address 127.0.0.1:43183)
0812 19:08:03.574328 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:03.574332 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:03.574482 (+ 150us) server_negotiation.cc:407] Connection header received
0812 19:08:03.574601 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:03.574604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:03.574653 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:03.574727 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:03.575117 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:03.575762 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:03.576575 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:03.576944 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:03.577062 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:03.577998 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:03.578002 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:03.578004 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:08:03.578019 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:03.578050 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:08:03.578063 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:03.578141 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:03.578490 (+ 349us) server_negotiation.cc:299] Negotiation successful
0812 19:08:03.578527 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 19:08:03.580067 19401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98477a7a3389496189be5e82299bd4ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98477a7a3389496189be5e82299bd4ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98477a7a3389496189be5e82299bd4ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43183 } } }
I20260812 19:08:03.580204 19401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98477a7a3389496189be5e82299bd4ca [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:03.580580 19400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98477a7a3389496189be5e82299bd4ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98477a7a3389496189be5e82299bd4ca. Latest consensus state: current_term: 1 leader_uuid: "98477a7a3389496189be5e82299bd4ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98477a7a3389496189be5e82299bd4ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43183 } } }
I20260812 19:08:03.580700 19400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98477a7a3389496189be5e82299bd4ca [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:03.580565 19415 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:08:03.584108 19415 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:08:03.589666 19415 catalog_manager.cc:1347] Generated new cluster ID: 646b1744298f49d28b02144529fb2f94
I20260812 19:08:03.589747 19415 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:08:03.595659 19415 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:08:03.597003 19415 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:08:03.603945 19415 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98477a7a3389496189be5e82299bd4ca: Generated new TSK 0
I20260812 19:08:03.604658 19415 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:08:03.616730 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 19305
W20260812 19:08:03.627991 19287 connection.cc:535] client connection to 127.0.0.1:43183 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43183): Connection reset by peer (error 104)
2026-08-12T19:08:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1437 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:08:03 dist-test-slave-49h3 krb5kdc[19427](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:08:03 dist-test-slave-49h3 krb5kdc[19427](info): set up 2 sockets
Aug 12 19:08:03 dist-test-slave-49h3 krb5kdc[19427](info): commencing operation
krb5kdc: starting...
W20260812 19:08:04.672421 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:08:04 dist-test-slave-49h3 krb5kdc[19427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:08:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:08:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 19:08:04.800194 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38845
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:08:04.914111 19449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:08:04.914425 19449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:08:04.914487 19449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:08:04.918571 19449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:08:04.918648 19449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:08:04.918676 19449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:08:04.918699 19449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:08:04.918727 19449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:08:04.924413 19449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:08:04.925623 19449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:08:04.926705 19449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:08:04.935338 19459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:08:04.935339 19456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:04.935407 19449 server_base.cc:1034] running on GCE node
W20260812 19:08:04.935338 19457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:04.936205 19449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:08:04.937417 19449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:08:04.938637 19449 hybrid_clock.cc:648] HybridClock initialized: now 1786561684938615 us; error 36 us; skew 500 ppm
Aug 12 19:08:04 dist-test-slave-49h3 krb5kdc[19427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561684, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:08:04.947027 19449 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 19:08:04.948071 19449 webserver.cc:460] Webserver started at http://127.0.0.1:38945/ using document root <none> and password file <none>
I20260812 19:08:04.948681 19449 fs_manager.cc:359] Metadata directory not provided
I20260812 19:08:04.948761 19449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:08:04.948983 19449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:08:04.951195 19449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "0f8a719168f64c5caa12ffa7962084ac"
format_stamp: "Formatted at 2026-08-12 19:08:04 on dist-test-slave-49h3"
server_key: "552395ad1f875c2a490387a546892c2b"
server_key_iv: "0b6912e2e567cbc181a9ff84bc1ea895"
server_key_version: "encryptionkey@0"
I20260812 19:08:04.951778 19449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "0f8a719168f64c5caa12ffa7962084ac"
format_stamp: "Formatted at 2026-08-12 19:08:04 on dist-test-slave-49h3"
server_key: "552395ad1f875c2a490387a546892c2b"
server_key_iv: "0b6912e2e567cbc181a9ff84bc1ea895"
server_key_version: "encryptionkey@0"
I20260812 19:08:04.955677 19449 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:08:04.958150 19468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:08:04.959410 19449 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:08:04.959539 19449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "0f8a719168f64c5caa12ffa7962084ac"
format_stamp: "Formatted at 2026-08-12 19:08:04 on dist-test-slave-49h3"
server_key: "552395ad1f875c2a490387a546892c2b"
server_key_iv: "0b6912e2e567cbc181a9ff84bc1ea895"
server_key_version: "encryptionkey@0"
I20260812 19:08:04.959640 19449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:08:04.964452 19449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:08:04.967724 19449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:08:04.967914 19449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:08:04.974973 19449 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38845
I20260812 19:08:04.975028 19532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38845 every 8 connection(s)
I20260812 19:08:04.975965 19449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:08:04.977370 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 19449
I20260812 19:08:04.977521 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:08:04.977809 26120 external_mini_cluster.cc:1428] Setting key 7f09bf8735ad76006329ad8f6ca30601
I20260812 19:08:04.979092 19533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:08:04.983021 19536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:04.979210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:60145 (local address 127.0.0.1:38845)
0812 19:08:04.979657 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:04.979672 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:04.979714 (+ 42us) server_negotiation.cc:407] Connection header received
0812 19:08:04.980360 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:04.980378 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:04.980661 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:04.980980 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:04.981814 (+ 834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:04.981819 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:04.981832 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:08:04.981842 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:04.981902 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:08:04.981922 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:04.982055 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:04.982126 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 19:08:04.982296 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":138,"threads_started":1}
I20260812 19:08:04.985533 19533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f8a719168f64c5caa12ffa7962084ac: Bootstrap starting.
I20260812 19:08:04.986116 26120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:08:04.988078 19533 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f8a719168f64c5caa12ffa7962084ac: Neither blocks nor log segments found. Creating new log.
I20260812 19:08:04.989076 19533 log.cc:824] T 00000000000000000000000000000000 P 0f8a719168f64c5caa12ffa7962084ac: Log is configured to *not* fsync() on all Append() calls
I20260812 19:08:04.989902 19536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:04.988153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:57379 (local address 127.0.0.1:38845)
0812 19:08:04.988288 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:04.988292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:04.988479 (+ 187us) server_negotiation.cc:407] Connection header received
0812 19:08:04.988609 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:04.988611 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:04.988643 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:04.988711 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:04.989490 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:04.989494 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:04.989497 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:08:04.989510 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:04.989539 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:08:04.989550 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:04.989624 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:04.989748 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 19:08:04.989783 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 19:08:04.991581 19533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f8a719168f64c5caa12ffa7962084ac: No bootstrap required, opened a new log
I20260812 19:08:04.995569 19533 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f8a719168f64c5caa12ffa7962084ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f8a719168f64c5caa12ffa7962084ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38845 } }
I20260812 19:08:04.995823 19533 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f8a719168f64c5caa12ffa7962084ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:08:04.995875 19533 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f8a719168f64c5caa12ffa7962084ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f8a719168f64c5caa12ffa7962084ac, State: Initialized, Role: FOLLOWER
I20260812 19:08:04.996292 19533 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f8a719168f64c5caa12ffa7962084ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f8a719168f64c5caa12ffa7962084ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38845 } }
I20260812 19:08:04.996424 19533 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f8a719168f64c5caa12ffa7962084ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:08:04.996492 19533 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f8a719168f64c5caa12ffa7962084ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:08:04.996596 19533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f8a719168f64c5caa12ffa7962084ac [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:08:04.997650 19533 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f8a719168f64c5caa12ffa7962084ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f8a719168f64c5caa12ffa7962084ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38845 } }
I20260812 19:08:04.997989 19533 leader_election.cc:302] T 00000000000000000000000000000000 P 0f8a719168f64c5caa12ffa7962084ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f8a719168f64c5caa12ffa7962084ac; no voters:
I20260812 19:08:04.998380 19533 leader_election.cc:288] T 00000000000000000000000000000000 P 0f8a719168f64c5caa12ffa7962084ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:08:04.998498 19546 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f8a719168f64c5caa12ffa7962084ac [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:08:04.998747 19546 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f8a719168f64c5caa12ffa7962084ac [term 1 LEADER]: Becoming Leader. State: Replica: 0f8a719168f64c5caa12ffa7962084ac, State: Running, Role: LEADER
I20260812 19:08:04.999136 19546 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f8a719168f64c5caa12ffa7962084ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f8a719168f64c5caa12ffa7962084ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38845 } }
I20260812 19:08:04.999646 19533 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f8a719168f64c5caa12ffa7962084ac [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:08:05.000764 19547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f8a719168f64c5caa12ffa7962084ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f8a719168f64c5caa12ffa7962084ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f8a719168f64c5caa12ffa7962084ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38845 } } }
I20260812 19:08:05.000909 19547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f8a719168f64c5caa12ffa7962084ac [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:05.001099 19548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f8a719168f64c5caa12ffa7962084ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f8a719168f64c5caa12ffa7962084ac. Latest consensus state: current_term: 1 leader_uuid: "0f8a719168f64c5caa12ffa7962084ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f8a719168f64c5caa12ffa7962084ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38845 } } }
I20260812 19:08:05.001258 19548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f8a719168f64c5caa12ffa7962084ac [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:05.001343 19559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:08:05.004765 19559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:08:05.010941 19559 catalog_manager.cc:1347] Generated new cluster ID: 7b3198fcbe8f4669808e0f4418a8b50a
I20260812 19:08:05.011034 19559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:08:05.028086 19559 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:08:05.028416 19559 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:08:05.033360 19559 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f8a719168f64c5caa12ffa7962084ac: Generated new TSK 0
I20260812 19:08:05.034183 19559 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:08:05.047475 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 19449
2026-08-12T19:08:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1429 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:08:05 dist-test-slave-49h3 krb5kdc[19576](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:08:05 dist-test-slave-49h3 krb5kdc[19576](info): set up 2 sockets
Aug 12 19:08:05 dist-test-slave-49h3 krb5kdc[19576](info): commencing operation
krb5kdc: starting...
W20260812 19:08:06.106685 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:08:06 dist-test-slave-49h3 krb5kdc[19576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:08:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:08:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 19:08:06.234480 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41141
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:08:06.347452 19591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:08:06.347741 19591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:08:06.347790 19591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:08:06.352077 19591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:08:06.352174 19591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:08:06.352193 19591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:08:06.352209 19591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:08:06.352223 19591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:08:06.357873 19591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41141
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:08:06.358990 19591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:08:06.360008 19591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:08:06.368206 19598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:08:06.368221 19601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:06.368261 19591 server_base.cc:1034] running on GCE node
W20260812 19:08:06.368219 19599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:06.368932 19591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:08:06.370066 19591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:08:06.371248 19591 hybrid_clock.cc:648] HybridClock initialized: now 1786561686371219 us; error 39 us; skew 500 ppm
Aug 12 19:08:06 dist-test-slave-49h3 krb5kdc[19576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561686, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:08:06.373889 19591 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 19:08:06.374773 19591 webserver.cc:460] Webserver started at http://127.0.0.1:40229/ using document root <none> and password file <none>
I20260812 19:08:06.375305 19591 fs_manager.cc:359] Metadata directory not provided
I20260812 19:08:06.375372 19591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:08:06.375561 19591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:08:06.377628 19591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "ae1e581ddad7402ca7ea19b289e28412"
format_stamp: "Formatted at 2026-08-12 19:08:06 on dist-test-slave-49h3"
server_key: "9009a8c850fc85e354b2a0d439714183"
server_key_iv: "17ef8b8d12043ab208c05638fe2ae09c"
server_key_version: "encryptionkey@0"
I20260812 19:08:06.378181 19591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "ae1e581ddad7402ca7ea19b289e28412"
format_stamp: "Formatted at 2026-08-12 19:08:06 on dist-test-slave-49h3"
server_key: "9009a8c850fc85e354b2a0d439714183"
server_key_iv: "17ef8b8d12043ab208c05638fe2ae09c"
server_key_version: "encryptionkey@0"
I20260812 19:08:06.382292 19591 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 19:08:06.384771 19610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:08:06.386037 19591 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:08:06.386173 19591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "ae1e581ddad7402ca7ea19b289e28412"
format_stamp: "Formatted at 2026-08-12 19:08:06 on dist-test-slave-49h3"
server_key: "9009a8c850fc85e354b2a0d439714183"
server_key_iv: "17ef8b8d12043ab208c05638fe2ae09c"
server_key_version: "encryptionkey@0"
I20260812 19:08:06.386287 19591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:08:06.398042 19591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:08:06.401206 19591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:08:06.401391 19591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:08:06.408645 19591 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41141
I20260812 19:08:06.408660 19674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41141 every 8 connection(s)
I20260812 19:08:06.409673 19591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:08:06.411175 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 19591
I20260812 19:08:06.411322 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:08:06.411620 26120 external_mini_cluster.cc:1428] Setting key ba2382e27ad6afc97e988afe135b6ba9
I20260812 19:08:06.413077 19675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:08:06.420087 19675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae1e581ddad7402ca7ea19b289e28412: Bootstrap starting.
Aug 12 19:08:06 dist-test-slave-49h3 krb5kdc[19576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 19:08:06.422803 19675 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae1e581ddad7402ca7ea19b289e28412: Neither blocks nor log segments found. Creating new log.
I20260812 19:08:06.423759 19675 log.cc:824] T 00000000000000000000000000000000 P ae1e581ddad7402ca7ea19b289e28412: Log is configured to *not* fsync() on all Append() calls
I20260812 19:08:06.426252 19675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae1e581ddad7402ca7ea19b289e28412: No bootstrap required, opened a new log
I20260812 19:08:06.427055 19678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:06.412991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:57967 (local address 127.0.0.1:41141)
0812 19:08:06.413510 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:06.413532 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:06.413575 (+ 43us) server_negotiation.cc:407] Connection header received
0812 19:08:06.414413 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:06.414437 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:06.414882 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:06.415254 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:06.415957 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:06.417517 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:06.418383 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:06.419099 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:06.419274 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:06.422048 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:06.422056 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:06.422073 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:06.422108 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:06.424011 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:06.424559 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:06.424566 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:06.424573 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:06.424705 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:06.425036 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:06.425038 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:06.425039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:06.425474 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:06.425697 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:06.425991 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 19:08:06.426241 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":141,"threads_started":1}
I20260812 19:08:06.430238 26120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:08:06.430450 19675 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae1e581ddad7402ca7ea19b289e28412 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae1e581ddad7402ca7ea19b289e28412" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41141 } }
I20260812 19:08:06.430693 19675 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae1e581ddad7402ca7ea19b289e28412 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:08:06.430763 19675 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae1e581ddad7402ca7ea19b289e28412 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae1e581ddad7402ca7ea19b289e28412, State: Initialized, Role: FOLLOWER
I20260812 19:08:06.431198 19675 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae1e581ddad7402ca7ea19b289e28412 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae1e581ddad7402ca7ea19b289e28412" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41141 } }
I20260812 19:08:06.431337 19675 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae1e581ddad7402ca7ea19b289e28412 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:08:06.431434 19675 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae1e581ddad7402ca7ea19b289e28412 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:08:06.431548 19675 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae1e581ddad7402ca7ea19b289e28412 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:08:06.432700 19675 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae1e581ddad7402ca7ea19b289e28412 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae1e581ddad7402ca7ea19b289e28412" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41141 } }
I20260812 19:08:06.433085 19675 leader_election.cc:302] T 00000000000000000000000000000000 P ae1e581ddad7402ca7ea19b289e28412 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae1e581ddad7402ca7ea19b289e28412; no voters:
I20260812 19:08:06.433537 19675 leader_election.cc:288] T 00000000000000000000000000000000 P ae1e581ddad7402ca7ea19b289e28412 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:08:06.433629 19686 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae1e581ddad7402ca7ea19b289e28412 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:08:06.433854 19686 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae1e581ddad7402ca7ea19b289e28412 [term 1 LEADER]: Becoming Leader. State: Replica: ae1e581ddad7402ca7ea19b289e28412, State: Running, Role: LEADER
I20260812 19:08:06.434208 19686 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae1e581ddad7402ca7ea19b289e28412 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae1e581ddad7402ca7ea19b289e28412" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41141 } }
I20260812 19:08:06.434928 19675 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae1e581ddad7402ca7ea19b289e28412 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:08:06.435758 19688 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae1e581ddad7402ca7ea19b289e28412 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae1e581ddad7402ca7ea19b289e28412" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae1e581ddad7402ca7ea19b289e28412" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41141 } } }
I20260812 19:08:06.435945 19688 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae1e581ddad7402ca7ea19b289e28412 [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:06.436244 19688 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae1e581ddad7402ca7ea19b289e28412 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae1e581ddad7402ca7ea19b289e28412. Latest consensus state: current_term: 1 leader_uuid: "ae1e581ddad7402ca7ea19b289e28412" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae1e581ddad7402ca7ea19b289e28412" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41141 } } }
I20260812 19:08:06.436383 19688 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae1e581ddad7402ca7ea19b289e28412 [sys.catalog]: This master's current role is: LEADER
W20260812 19:08:06.439759 19708 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ae1e581ddad7402ca7ea19b289e28412: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:08:06.439831 19708 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:08:06.439961 19698 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:08:06.440888 19698 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:08:06.443980 19678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:06.432320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:51757 (local address 127.0.0.1:41141)
0812 19:08:06.432434 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:06.432437 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:06.432556 (+ 119us) server_negotiation.cc:407] Connection header received
0812 19:08:06.432691 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:06.432693 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:06.432735 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:06.432788 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:06.433619 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:06.434369 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:06.435489 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:06.435844 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:06.435947 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:06.437845 (+ 1898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:06.437849 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:06.437852 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:06.437878 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:06.442233 (+ 4355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:06.442742 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:06.442745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:06.442746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:06.442825 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:06.443161 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:06.443164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:06.443165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:06.443346 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:06.443472 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:06.443739 (+ 267us) server_negotiation.cc:299] Negotiation successful
0812 19:08:06.443832 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 19:08:06.446944 19698 catalog_manager.cc:1347] Generated new cluster ID: b645162d0f1848f3932b7fc9c5fa8bbc
I20260812 19:08:06.447016 19698 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:08:06.465695 19698 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:08:06.466676 19698 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:08:06.476948 19698 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae1e581ddad7402ca7ea19b289e28412: Generated new TSK 0
I20260812 19:08:06.477741 19698 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:08:06.481304 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 19591
W20260812 19:08:06.491669 19574 connection.cc:535] client connection to 127.0.0.1:41141 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41141): Connection reset by peer (error 104)
2026-08-12T19:08:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1434 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:08:06 dist-test-slave-49h3 krb5kdc[19714](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:08:06 dist-test-slave-49h3 krb5kdc[19714](info): set up 2 sockets
Aug 12 19:08:06 dist-test-slave-49h3 krb5kdc[19714](info): commencing operation
krb5kdc: starting...
W20260812 19:08:07.536783 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:08:07 dist-test-slave-49h3 krb5kdc[19714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:08:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:08:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 19:08:07.662753 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32845
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:08:07.776998 19734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:08:07.777302 19734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:08:07.777345 19734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:08:07.781594 19734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:08:07.781668 19734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:08:07.781687 19734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:08:07.781723 19734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:08:07.781751 19734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:08:07.787354 19734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32845
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:08:07.788527 19734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:08:07.789680 19734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:08:07.798628 19734 server_base.cc:1034] running on GCE node
W20260812 19:08:07.798573 19740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:08:07.798569 19741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:08:07.798569 19744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:07.799505 19734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:08:07.800746 19734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:08:07.801910 19734 hybrid_clock.cc:648] HybridClock initialized: now 1786561687801888 us; error 37 us; skew 500 ppm
Aug 12 19:08:07 dist-test-slave-49h3 krb5kdc[19714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561687, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:08:07.804641 19734 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 19:08:07.805675 19734 webserver.cc:460] Webserver started at http://127.0.0.1:45647/ using document root <none> and password file <none>
I20260812 19:08:07.806280 19734 fs_manager.cc:359] Metadata directory not provided
I20260812 19:08:07.806356 19734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:08:07.806576 19734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:08:07.808770 19734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "73cc0be6be6948b59b9defdddc959607"
format_stamp: "Formatted at 2026-08-12 19:08:07 on dist-test-slave-49h3"
server_key: "baad77731bb27c6c2e100b998fca6cb2"
server_key_iv: "b647e3970f5cb9289a21b1771f7efda9"
server_key_version: "encryptionkey@0"
I20260812 19:08:07.809378 19734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "73cc0be6be6948b59b9defdddc959607"
format_stamp: "Formatted at 2026-08-12 19:08:07 on dist-test-slave-49h3"
server_key: "baad77731bb27c6c2e100b998fca6cb2"
server_key_iv: "b647e3970f5cb9289a21b1771f7efda9"
server_key_version: "encryptionkey@0"
I20260812 19:08:07.813421 19734 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 19:08:07.816015 19755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:08:07.817315 19734 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:08:07.817478 19734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "73cc0be6be6948b59b9defdddc959607"
format_stamp: "Formatted at 2026-08-12 19:08:07 on dist-test-slave-49h3"
server_key: "baad77731bb27c6c2e100b998fca6cb2"
server_key_iv: "b647e3970f5cb9289a21b1771f7efda9"
server_key_version: "encryptionkey@0"
I20260812 19:08:07.817600 19734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:08:07.853116 19734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:08:07.856084 19734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:08:07.856302 19734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:08:07.863510 19734 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32845
I20260812 19:08:07.863526 19817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32845 every 8 connection(s)
I20260812 19:08:07.864496 19734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:08:07.867408 19818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:08:07.870412 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 19734
I20260812 19:08:07.870568 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:08:07.870855 26120 external_mini_cluster.cc:1428] Setting key 90875d5931985646043a21b3a5e04698
I20260812 19:08:07.873721 19818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73cc0be6be6948b59b9defdddc959607: Bootstrap starting.
I20260812 19:08:07.876405 19818 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73cc0be6be6948b59b9defdddc959607: Neither blocks nor log segments found. Creating new log.
I20260812 19:08:07.877382 19818 log.cc:824] T 00000000000000000000000000000000 P 73cc0be6be6948b59b9defdddc959607: Log is configured to *not* fsync() on all Append() calls
I20260812 19:08:07.880350 19818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73cc0be6be6948b59b9defdddc959607: No bootstrap required, opened a new log
I20260812 19:08:07.880784 19824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:07.872313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:56549 (local address 127.0.0.1:32845)
0812 19:08:07.872754 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:07.872770 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:07.872807 (+ 37us) server_negotiation.cc:407] Connection header received
0812 19:08:07.873503 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:07.873524 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:07.873926 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:07.874228 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:07.874570 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:07.875739 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:07.876683 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:07.877362 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:07.877545 (+ 183us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:07.878599 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:07.878606 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:07.878624 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:08:07.879145 (+ 521us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:07.879218 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:08:07.879250 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:07.879381 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:07.879640 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 19:08:07.879791 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":128,"threads_started":1}
I20260812 19:08:07.884496 19818 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73cc0be6be6948b59b9defdddc959607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73cc0be6be6948b59b9defdddc959607" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32845 } }
I20260812 19:08:07.884737 19818 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73cc0be6be6948b59b9defdddc959607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:08:07.884774 19818 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73cc0be6be6948b59b9defdddc959607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73cc0be6be6948b59b9defdddc959607, State: Initialized, Role: FOLLOWER
I20260812 19:08:07.885164 19818 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73cc0be6be6948b59b9defdddc959607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73cc0be6be6948b59b9defdddc959607" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32845 } }
I20260812 19:08:07.885305 19818 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73cc0be6be6948b59b9defdddc959607 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:08:07.885368 19818 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73cc0be6be6948b59b9defdddc959607 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:08:07.885486 19818 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73cc0be6be6948b59b9defdddc959607 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:08:07.886591 19818 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73cc0be6be6948b59b9defdddc959607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73cc0be6be6948b59b9defdddc959607" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32845 } }
I20260812 19:08:07.886689 26120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:08:07.886962 19818 leader_election.cc:302] T 00000000000000000000000000000000 P 73cc0be6be6948b59b9defdddc959607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73cc0be6be6948b59b9defdddc959607; no voters:
I20260812 19:08:07.887338 19818 leader_election.cc:288] T 00000000000000000000000000000000 P 73cc0be6be6948b59b9defdddc959607 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:08:07.887483 19826 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73cc0be6be6948b59b9defdddc959607 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:08:07.887768 19826 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73cc0be6be6948b59b9defdddc959607 [term 1 LEADER]: Becoming Leader. State: Replica: 73cc0be6be6948b59b9defdddc959607, State: Running, Role: LEADER
I20260812 19:08:07.888147 19826 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73cc0be6be6948b59b9defdddc959607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73cc0be6be6948b59b9defdddc959607" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32845 } }
I20260812 19:08:07.888796 19818 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73cc0be6be6948b59b9defdddc959607 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:08:07.889904 19827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73cc0be6be6948b59b9defdddc959607 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73cc0be6be6948b59b9defdddc959607" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73cc0be6be6948b59b9defdddc959607" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32845 } } }
I20260812 19:08:07.890028 19827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73cc0be6be6948b59b9defdddc959607 [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:07.890252 19827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73cc0be6be6948b59b9defdddc959607 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73cc0be6be6948b59b9defdddc959607. Latest consensus state: current_term: 1 leader_uuid: "73cc0be6be6948b59b9defdddc959607" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73cc0be6be6948b59b9defdddc959607" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32845 } } }
I20260812 19:08:07.890328 19827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73cc0be6be6948b59b9defdddc959607 [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:07.894181 19824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:07.889019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:53149 (local address 127.0.0.1:32845)
0812 19:08:07.889129 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:07.889134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:07.889389 (+ 255us) server_negotiation.cc:407] Connection header received
0812 19:08:07.889629 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:07.889632 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:07.889689 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:07.889758 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:07.890218 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:07.890964 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:07.892064 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:07.892367 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:07.892464 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:07.893362 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:07.893365 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:07.893368 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:08:07.893382 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:07.893411 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:08:07.893424 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:07.893948 (+ 524us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:07.894002 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 19:08:07.894039 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260812 19:08:07.894610 19850 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 73cc0be6be6948b59b9defdddc959607: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:08:07.894765 19850 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:08:07.894879 19841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:08:07.895761 19841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:08:07.901623 19841 catalog_manager.cc:1347] Generated new cluster ID: f8175927fa524703a490b39ceb534a95
I20260812 19:08:07.901692 19841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:08:07.932753 19841 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:08:07.934229 19841 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:08:07.940577 19841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73cc0be6be6948b59b9defdddc959607: Generated new TSK 0
I20260812 19:08:07.941498 19841 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:08:07.956549 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 19734
W20260812 19:08:07.969280 19712 connection.cc:535] client connection to 127.0.0.1:32845 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32845): Connection reset by peer (error 104)
2026-08-12T19:08:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1478 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:08:08 dist-test-slave-49h3 krb5kdc[19855](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:08:08 dist-test-slave-49h3 krb5kdc[19855](info): set up 2 sockets
Aug 12 19:08:08 dist-test-slave-49h3 krb5kdc[19855](info): commencing operation
krb5kdc: starting...
W20260812 19:08:09.018486 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:08:09 dist-test-slave-49h3 krb5kdc[19855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:08:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:08:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 19:08:09.142627 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35273
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:08:09.255421 19870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:08:09.255708 19870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:08:09.255769 19870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:08:09.259773 19870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:08:09.259843 19870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:08:09.259871 19870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:08:09.259891 19870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:08:09.259923 19870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:08:09.265328 19870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35273
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561415988404-26120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:08:09.266494 19870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:08:09.267530 19870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:08:09.276381 19876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:08:09.276444 19875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:09.276474 19870 server_base.cc:1034] running on GCE node
W20260812 19:08:09.276409 19878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:09.277264 19870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:08:09.278479 19870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:08:09.279649 19870 hybrid_clock.cc:648] HybridClock initialized: now 1786561689279623 us; error 41 us; skew 500 ppm
Aug 12 19:08:09 dist-test-slave-49h3 krb5kdc[19855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561689, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:08:09.282493 19870 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 19:08:09.283474 19870 webserver.cc:460] Webserver started at http://127.0.0.1:38765/ using document root <none> and password file <none>
I20260812 19:08:09.284026 19870 fs_manager.cc:359] Metadata directory not provided
I20260812 19:08:09.284090 19870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:08:09.284273 19870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:08:09.286525 19870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "2c0102a16ee54dd390be97abbc32a53c"
format_stamp: "Formatted at 2026-08-12 19:08:09 on dist-test-slave-49h3"
server_key: "5139f7fdeeb77548609a2f5f115652a5"
server_key_iv: "41609340374c1dd5b94c02e793ed62ca"
server_key_version: "encryptionkey@0"
I20260812 19:08:09.287086 19870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "2c0102a16ee54dd390be97abbc32a53c"
format_stamp: "Formatted at 2026-08-12 19:08:09 on dist-test-slave-49h3"
server_key: "5139f7fdeeb77548609a2f5f115652a5"
server_key_iv: "41609340374c1dd5b94c02e793ed62ca"
server_key_version: "encryptionkey@0"
I20260812 19:08:09.290817 19870 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 19:08:09.293270 19892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:08:09.294838 19870 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:08:09.294951 19870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "2c0102a16ee54dd390be97abbc32a53c"
format_stamp: "Formatted at 2026-08-12 19:08:09 on dist-test-slave-49h3"
server_key: "5139f7fdeeb77548609a2f5f115652a5"
server_key_iv: "41609340374c1dd5b94c02e793ed62ca"
server_key_version: "encryptionkey@0"
I20260812 19:08:09.295051 19870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:08:09.299594 19870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:08:09.302444 19870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:08:09.302615 19870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:08:09.309417 19870 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35273
I20260812 19:08:09.309470 19953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35273 every 8 connection(s)
I20260812 19:08:09.310377 19870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:08:09.313050 19954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:08:09.318534 19954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c0102a16ee54dd390be97abbc32a53c: Bootstrap starting.
I20260812 19:08:09.319802 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 19870
I20260812 19:08:09.319937 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:08:09.320197 26120 external_mini_cluster.cc:1428] Setting key 7b13ddd7c49d5f624ab005753b7c788f
I20260812 19:08:09.321231 19954 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c0102a16ee54dd390be97abbc32a53c: Neither blocks nor log segments found. Creating new log.
I20260812 19:08:09.322137 19954 log.cc:824] T 00000000000000000000000000000000 P 2c0102a16ee54dd390be97abbc32a53c: Log is configured to *not* fsync() on all Append() calls
I20260812 19:08:09.324468 19954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c0102a16ee54dd390be97abbc32a53c: No bootstrap required, opened a new log
I20260812 19:08:09.325641 19957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:09.321329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:60493 (local address 127.0.0.1:35273)
0812 19:08:09.321867 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:09.321885 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:09.321930 (+ 45us) server_negotiation.cc:407] Connection header received
0812 19:08:09.322778 (+ 848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:09.322798 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:09.323128 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:09.323494 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:09.324153 (+ 659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:09.324158 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:09.324172 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:08:09.324183 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:09.324246 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:08:09.324273 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:09.324379 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:09.324479 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 19:08:09.324644 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":139,"threads_started":1}
I20260812 19:08:09.328624 26120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:08:09.328871 19954 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c0102a16ee54dd390be97abbc32a53c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c0102a16ee54dd390be97abbc32a53c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35273 } }
I20260812 19:08:09.329104 19954 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c0102a16ee54dd390be97abbc32a53c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:08:09.329156 19954 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c0102a16ee54dd390be97abbc32a53c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0102a16ee54dd390be97abbc32a53c, State: Initialized, Role: FOLLOWER
I20260812 19:08:09.329617 19954 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c0102a16ee54dd390be97abbc32a53c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c0102a16ee54dd390be97abbc32a53c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35273 } }
I20260812 19:08:09.329749 19954 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c0102a16ee54dd390be97abbc32a53c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:08:09.329841 19954 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c0102a16ee54dd390be97abbc32a53c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:08:09.329952 19954 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c0102a16ee54dd390be97abbc32a53c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:08:09.331116 19954 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c0102a16ee54dd390be97abbc32a53c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c0102a16ee54dd390be97abbc32a53c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35273 } }
I20260812 19:08:09.331554 19954 leader_election.cc:302] T 00000000000000000000000000000000 P 2c0102a16ee54dd390be97abbc32a53c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c0102a16ee54dd390be97abbc32a53c; no voters:
I20260812 19:08:09.331944 19954 leader_election.cc:288] T 00000000000000000000000000000000 P 2c0102a16ee54dd390be97abbc32a53c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:08:09.332049 19966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c0102a16ee54dd390be97abbc32a53c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:08:09.332298 19966 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c0102a16ee54dd390be97abbc32a53c [term 1 LEADER]: Becoming Leader. State: Replica: 2c0102a16ee54dd390be97abbc32a53c, State: Running, Role: LEADER
I20260812 19:08:09.332562 19957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:09.330720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.148:40973 (local address 127.0.0.1:35273)
0812 19:08:09.330832 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:09.330837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:09.331018 (+ 181us) server_negotiation.cc:407] Connection header received
0812 19:08:09.331134 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:09.331136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:09.331172 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:09.331247 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:09.332096 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:09.332100 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:09.332103 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:08:09.332118 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:09.332151 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:08:09.332165 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:09.332255 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:09.332407 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 19:08:09.332448 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 19:08:09.332619 19966 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c0102a16ee54dd390be97abbc32a53c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c0102a16ee54dd390be97abbc32a53c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35273 } }
I20260812 19:08:09.333274 19954 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c0102a16ee54dd390be97abbc32a53c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:08:09.334025 19966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c0102a16ee54dd390be97abbc32a53c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c0102a16ee54dd390be97abbc32a53c. Latest consensus state: current_term: 1 leader_uuid: "2c0102a16ee54dd390be97abbc32a53c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c0102a16ee54dd390be97abbc32a53c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35273 } } }
I20260812 19:08:09.334156 19966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c0102a16ee54dd390be97abbc32a53c [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:09.334452 19968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c0102a16ee54dd390be97abbc32a53c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c0102a16ee54dd390be97abbc32a53c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c0102a16ee54dd390be97abbc32a53c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35273 } } }
I20260812 19:08:09.334537 19968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c0102a16ee54dd390be97abbc32a53c [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:09.334774 19976 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:08:09.338119 19976 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:08:09.343883 19976 catalog_manager.cc:1347] Generated new cluster ID: 8ea26139061c47309a06fb857498b5bd
I20260812 19:08:09.343972 19976 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:08:09.358889 19976 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:08:09.359229 19976 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:08:09.367367 19976 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c0102a16ee54dd390be97abbc32a53c: Generated new TSK 0
I20260812 19:08:09.368039 19976 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:08:09.374347 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 19870
2026-08-12T19:08:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1411 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:08:09 dist-test-slave-49h3 krb5kdc[19990](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:08:09 dist-test-slave-49h3 krb5kdc[19990](info): set up 2 sockets
Aug 12 19:08:09 dist-test-slave-49h3 krb5kdc[19990](info): commencing operation
krb5kdc: starting...
W20260812 19:08:10.430856 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:08:10 dist-test-slave-49h3 krb5kdc[19990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:08:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:08:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:08:10.556540 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:45759
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:35265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:45759
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:08:10.670938 20005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:08:10.671250 20005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:08:10.671294 20005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:08:10.675433 20005 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:08:10.675519 20005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:08:10.675539 20005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:08:10.675554 20005 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:08:10.675568 20005 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:08:10.681291 20005 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:35265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:45759
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:45759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:08:10.682477 20005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:08:10.683552 20005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:08:10.692337 20013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:10.692548 20005 server_base.cc:1034] running on GCE node
W20260812 19:08:10.692394 20012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:08:10.692414 20015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:10.693289 20005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:08:10.694475 20005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:08:10.695638 20005 hybrid_clock.cc:648] HybridClock initialized: now 1786561690695623 us; error 34 us; skew 500 ppm
Aug 12 19:08:10 dist-test-slave-49h3 krb5kdc[19990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561690, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:08:10.703702 20005 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:08:10.704744 20005 webserver.cc:460] Webserver started at http://127.25.130.62:36617/ using document root <none> and password file <none>
I20260812 19:08:10.705307 20005 fs_manager.cc:359] Metadata directory not provided
I20260812 19:08:10.705372 20005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:08:10.705626 20005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:08:10.707849 20005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "64f476ace84a4d029042802c0521d532"
format_stamp: "Formatted at 2026-08-12 19:08:10 on dist-test-slave-49h3"
server_key: "904f2c3c9866cda2f9fca0a404e56913"
server_key_iv: "61396eaa737b17654d42e7e98c7955e8"
server_key_version: "encryptionkey@0"
I20260812 19:08:10.708397 20005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "64f476ace84a4d029042802c0521d532"
format_stamp: "Formatted at 2026-08-12 19:08:10 on dist-test-slave-49h3"
server_key: "904f2c3c9866cda2f9fca0a404e56913"
server_key_iv: "61396eaa737b17654d42e7e98c7955e8"
server_key_version: "encryptionkey@0"
I20260812 19:08:10.712323 20005 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 19:08:10.714886 20023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:08:10.716169 20005 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 19:08:10.716288 20005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "64f476ace84a4d029042802c0521d532"
format_stamp: "Formatted at 2026-08-12 19:08:10 on dist-test-slave-49h3"
server_key: "904f2c3c9866cda2f9fca0a404e56913"
server_key_iv: "61396eaa737b17654d42e7e98c7955e8"
server_key_version: "encryptionkey@0"
I20260812 19:08:10.716398 20005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:08:10.730492 20005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:08:10.733846 20005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:08:10.734017 20005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:08:10.741254 20005 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:45759
I20260812 19:08:10.741266 20091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:45759 every 8 connection(s)
I20260812 19:08:10.742334 20005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:08:10.743912 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 20005
I20260812 19:08:10.744055 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:08:10.744374 26120 external_mini_cluster.cc:1428] Setting key ba650616b24ce788d3d68a8e2ecf4339
I20260812 19:08:10.745685 20092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:08:10.751470 20092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64f476ace84a4d029042802c0521d532: Bootstrap starting.
Aug 12 19:08:10 dist-test-slave-49h3 krb5kdc[19990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:08:10.754205 20092 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64f476ace84a4d029042802c0521d532: Neither blocks nor log segments found. Creating new log.
I20260812 19:08:10.755106 20092 log.cc:824] T 00000000000000000000000000000000 P 64f476ace84a4d029042802c0521d532: Log is configured to *not* fsync() on all Append() calls
I20260812 19:08:10.757097 20092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64f476ace84a4d029042802c0521d532: No bootstrap required, opened a new log
I20260812 19:08:10.758793 20095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:10.745917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50813 (local address 127.25.130.62:45759)
0812 19:08:10.746410 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:10.746426 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:10.746465 (+ 39us) server_negotiation.cc:407] Connection header received
0812 19:08:10.747107 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:10.747134 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:10.747470 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:10.747783 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:10.748601 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:10.749680 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:10.750518 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:10.751007 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:10.751166 (+ 159us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:10.754059 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:10.754065 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:10.754079 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:10.754111 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:10.755833 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:10.756361 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:10.756366 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:10.756371 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:10.756487 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:10.756819 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:10.756821 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:10.756823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:10.757244 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:10.757479 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:10.757821 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 19:08:10.758043 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":168,"threads_started":1}
I20260812 19:08:10.761152 20092 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64f476ace84a4d029042802c0521d532 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f476ace84a4d029042802c0521d532" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 45759 } }
I20260812 19:08:10.761439 20092 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64f476ace84a4d029042802c0521d532 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:08:10.761525 20092 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64f476ace84a4d029042802c0521d532 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f476ace84a4d029042802c0521d532, State: Initialized, Role: FOLLOWER
I20260812 19:08:10.762045 26120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:08:10.762013 20092 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64f476ace84a4d029042802c0521d532 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f476ace84a4d029042802c0521d532" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 45759 } }
I20260812 19:08:10.762153 20092 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64f476ace84a4d029042802c0521d532 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:08:10.762220 20092 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64f476ace84a4d029042802c0521d532 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:08:10.762318 20092 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64f476ace84a4d029042802c0521d532 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:08:10.763336 20092 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64f476ace84a4d029042802c0521d532 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f476ace84a4d029042802c0521d532" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 45759 } }
I20260812 19:08:10.763690 20092 leader_election.cc:302] T 00000000000000000000000000000000 P 64f476ace84a4d029042802c0521d532 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64f476ace84a4d029042802c0521d532; no voters:
I20260812 19:08:10.764009 20092 leader_election.cc:288] T 00000000000000000000000000000000 P 64f476ace84a4d029042802c0521d532 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:08:10.764086 20102 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64f476ace84a4d029042802c0521d532 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:08:10.764264 20102 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64f476ace84a4d029042802c0521d532 [term 1 LEADER]: Becoming Leader. State: Replica: 64f476ace84a4d029042802c0521d532, State: Running, Role: LEADER
I20260812 19:08:10.764562 20102 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64f476ace84a4d029042802c0521d532 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f476ace84a4d029042802c0521d532" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 45759 } }
I20260812 19:08:10.765343 20092 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64f476ace84a4d029042802c0521d532 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:08:10.766261 20104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64f476ace84a4d029042802c0521d532 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64f476ace84a4d029042802c0521d532" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f476ace84a4d029042802c0521d532" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 45759 } } }
I20260812 19:08:10.766391 20104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64f476ace84a4d029042802c0521d532 [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:10.768162 20105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64f476ace84a4d029042802c0521d532 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64f476ace84a4d029042802c0521d532. Latest consensus state: current_term: 1 leader_uuid: "64f476ace84a4d029042802c0521d532" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f476ace84a4d029042802c0521d532" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 45759 } } }
I20260812 19:08:10.768254 20105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64f476ace84a4d029042802c0521d532 [sys.catalog]: This master's current role is: LEADER
W20260812 19:08:10.771214 20120 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 64f476ace84a4d029042802c0521d532: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:08:10.771312 20120 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:08:10.771430 20113 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:08:10.772423 20113 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:08:10.773495 20095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:10.764103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42145 (local address 127.25.130.62:45759)
0812 19:08:10.764210 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:10.764214 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:10.764595 (+ 381us) server_negotiation.cc:407] Connection header received
0812 19:08:10.764854 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:10.764857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:10.764900 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:10.764958 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:10.765671 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:10.766482 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:10.767683 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:10.768009 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:10.768069 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:10.769951 (+ 1882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:10.769956 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:10.769958 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:10.769986 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:10.771662 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:10.772136 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:10.772139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:10.772141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:10.772225 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:10.772608 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:10.772611 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:10.772612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:10.772795 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:10.772928 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:10.773213 (+ 285us) server_negotiation.cc:299] Negotiation successful
0812 19:08:10.773303 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 19:08:10.779136 20113 catalog_manager.cc:1347] Generated new cluster ID: 7a473c0b5f794625935dfaedd7632416
I20260812 19:08:10.779212 20113 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:08:10.790611 20113 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:08:10.791942 20113 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:08:10.804932 20113 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64f476ace84a4d029042802c0521d532: Generated new TSK 0
I20260812 19:08:10.805958 20113 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:08:10.809983 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 20005
W20260812 19:08:10.818370 19988 connection.cc:535] client connection to 127.25.130.62:45759 recv error: Network error: failed to read from TLS socket (remote: 127.25.130.62:45759): Connection reset by peer (error 104)
2026-08-12T19:08:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1436 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:08:10 dist-test-slave-49h3 krb5kdc[20127](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:08:10 dist-test-slave-49h3 krb5kdc[20127](info): set up 2 sockets
Aug 12 19:08:10 dist-test-slave-49h3 krb5kdc[20127](info): commencing operation
krb5kdc: starting...
W20260812 19:08:11.861578 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:08:11 dist-test-slave-49h3 krb5kdc[20127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:08:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:08:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:08:11.988794 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:41319
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:39483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:41319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:08:12.105688 20142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:08:12.105974 20142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:08:12.106036 20142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:08:12.110116 20142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:08:12.110183 20142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:08:12.110200 20142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:08:12.110215 20142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:08:12.110229 20142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:08:12.115672 20142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:39483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:41319
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:41319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:08:12.116701 20142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:08:12.117698 20142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:08:12.125924 20148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:08:12.125885 20151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:12.125902 20142 server_base.cc:1034] running on GCE node
W20260812 19:08:12.125895 20147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:12.126714 20142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:08:12.127804 20142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:08:12.128966 20142 hybrid_clock.cc:648] HybridClock initialized: now 1786561692128954 us; error 32 us; skew 500 ppm
Aug 12 19:08:12 dist-test-slave-49h3 krb5kdc[20127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561692, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:08:12.131701 20142 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:08:12.132598 20142 webserver.cc:460] Webserver started at http://127.25.130.62:40659/ using document root <none> and password file <none>
I20260812 19:08:12.133140 20142 fs_manager.cc:359] Metadata directory not provided
I20260812 19:08:12.133211 20142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:08:12.133426 20142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:08:12.135545 20142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "9658500430fa475b8a0d1c9785ff0129"
format_stamp: "Formatted at 2026-08-12 19:08:12 on dist-test-slave-49h3"
server_key: "8666ed61a5111d0718d38763972f14c4"
server_key_iv: "6c61abe4a464ddb375614382984a2612"
server_key_version: "encryptionkey@0"
I20260812 19:08:12.136168 20142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "9658500430fa475b8a0d1c9785ff0129"
format_stamp: "Formatted at 2026-08-12 19:08:12 on dist-test-slave-49h3"
server_key: "8666ed61a5111d0718d38763972f14c4"
server_key_iv: "6c61abe4a464ddb375614382984a2612"
server_key_version: "encryptionkey@0"
I20260812 19:08:12.140045 20142 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:08:12.142550 20158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:08:12.143822 20142 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 19:08:12.143953 20142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "9658500430fa475b8a0d1c9785ff0129"
format_stamp: "Formatted at 2026-08-12 19:08:12 on dist-test-slave-49h3"
server_key: "8666ed61a5111d0718d38763972f14c4"
server_key_iv: "6c61abe4a464ddb375614382984a2612"
server_key_version: "encryptionkey@0"
I20260812 19:08:12.144061 20142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:08:12.175796 20142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:08:12.179092 20142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:08:12.179294 20142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:08:12.186553 20142 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:41319
I20260812 19:08:12.186594 20221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:41319 every 8 connection(s)
I20260812 19:08:12.187515 20142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:08:12.190382 20222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:08:12.195962 20222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9658500430fa475b8a0d1c9785ff0129: Bootstrap starting.
I20260812 19:08:12.196832 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 20142
I20260812 19:08:12.196959 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:08:12.197186 26120 external_mini_cluster.cc:1428] Setting key ac4cc74b8f3b372d32f9ad49bd053eee
I20260812 19:08:12.198751 20222 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9658500430fa475b8a0d1c9785ff0129: Neither blocks nor log segments found. Creating new log.
I20260812 19:08:12.199478 20222 log.cc:824] T 00000000000000000000000000000000 P 9658500430fa475b8a0d1c9785ff0129: Log is configured to *not* fsync() on all Append() calls
I20260812 19:08:12.201581 20222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9658500430fa475b8a0d1c9785ff0129: No bootstrap required, opened a new log
I20260812 19:08:12.205739 20222 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9658500430fa475b8a0d1c9785ff0129 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9658500430fa475b8a0d1c9785ff0129" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 41319 } }
I20260812 19:08:12.205926 20227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:12.198225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35131 (local address 127.25.130.62:41319)
0812 19:08:12.198668 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:12.198682 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:12.198718 (+ 36us) server_negotiation.cc:407] Connection header received
0812 19:08:12.199378 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:12.199395 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:12.199768 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:12.200145 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:12.200519 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:12.201515 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:12.202682 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:12.203160 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:12.203277 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:12.204174 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:12.204179 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:12.204193 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:08:12.204578 (+ 385us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:12.204635 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:08:12.204662 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:12.204763 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:12.204937 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 19:08:12.205071 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":142,"threads_started":1}
I20260812 19:08:12.205971 20222 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9658500430fa475b8a0d1c9785ff0129 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:08:12.206008 20222 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9658500430fa475b8a0d1c9785ff0129 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9658500430fa475b8a0d1c9785ff0129, State: Initialized, Role: FOLLOWER
I20260812 19:08:12.206483 20222 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9658500430fa475b8a0d1c9785ff0129 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9658500430fa475b8a0d1c9785ff0129" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 41319 } }
I20260812 19:08:12.206621 20222 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9658500430fa475b8a0d1c9785ff0129 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:08:12.206681 20222 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9658500430fa475b8a0d1c9785ff0129 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:08:12.206774 20222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9658500430fa475b8a0d1c9785ff0129 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:08:12.207955 20222 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9658500430fa475b8a0d1c9785ff0129 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9658500430fa475b8a0d1c9785ff0129" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 41319 } }
I20260812 19:08:12.208333 20222 leader_election.cc:302] T 00000000000000000000000000000000 P 9658500430fa475b8a0d1c9785ff0129 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9658500430fa475b8a0d1c9785ff0129; no voters:
I20260812 19:08:12.208654 20222 leader_election.cc:288] T 00000000000000000000000000000000 P 9658500430fa475b8a0d1c9785ff0129 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:08:12.208761 20229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9658500430fa475b8a0d1c9785ff0129 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:08:12.209033 20229 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9658500430fa475b8a0d1c9785ff0129 [term 1 LEADER]: Becoming Leader. State: Replica: 9658500430fa475b8a0d1c9785ff0129, State: Running, Role: LEADER
I20260812 19:08:12.209048 26120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:08:12.209332 20229 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9658500430fa475b8a0d1c9785ff0129 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9658500430fa475b8a0d1c9785ff0129" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 41319 } }
I20260812 19:08:12.210073 20222 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9658500430fa475b8a0d1c9785ff0129 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:08:12.210464 20229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9658500430fa475b8a0d1c9785ff0129 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9658500430fa475b8a0d1c9785ff0129. Latest consensus state: current_term: 1 leader_uuid: "9658500430fa475b8a0d1c9785ff0129" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9658500430fa475b8a0d1c9785ff0129" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 41319 } } }
I20260812 19:08:12.210664 20229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9658500430fa475b8a0d1c9785ff0129 [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:12.210875 20231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9658500430fa475b8a0d1c9785ff0129 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9658500430fa475b8a0d1c9785ff0129" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9658500430fa475b8a0d1c9785ff0129" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 41319 } } }
I20260812 19:08:12.211017 20231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9658500430fa475b8a0d1c9785ff0129 [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:12.211103 20242 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:08:12.214462 20242 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:08:12.216892 20227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:12.211410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55167 (local address 127.25.130.62:41319)
0812 19:08:12.211520 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:12.211524 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:12.212243 (+ 719us) server_negotiation.cc:407] Connection header received
0812 19:08:12.212356 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:12.212359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:12.212408 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:12.212481 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:12.212925 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:12.213775 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:12.214933 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:12.215376 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:12.215448 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:12.216433 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:12.216438 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:12.216440 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:08:12.216455 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:12.216482 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:08:12.216493 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:12.216566 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:12.216709 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 19:08:12.216748 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 19:08:12.220916 20242 catalog_manager.cc:1347] Generated new cluster ID: a3d41518fe494b508cb456def5a444fc
I20260812 19:08:12.220997 20242 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:08:12.239248 20242 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:08:12.240595 20242 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:08:12.249112 20242 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9658500430fa475b8a0d1c9785ff0129: Generated new TSK 0
I20260812 19:08:12.249867 20242 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:08:12.254335 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 20142
W20260812 19:08:12.262545 20125 connection.cc:535] client connection to 127.25.130.62:41319 recv error: Network error: failed to read from TLS socket (remote: 127.25.130.62:41319): Connection reset by peer (error 104)
2026-08-12T19:08:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1443 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:08:12 dist-test-slave-49h3 krb5kdc[20261](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:08:12 dist-test-slave-49h3 krb5kdc[20261](info): set up 2 sockets
Aug 12 19:08:12 dist-test-slave-49h3 krb5kdc[20261](info): commencing operation
krb5kdc: starting...
W20260812 19:08:13.310767 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:08:13 dist-test-slave-49h3 krb5kdc[20261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:08:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:08:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:08:13.436002 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:38443
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:40911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:38443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:08:13.548425 20275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:08:13.548749 20275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:08:13.548815 20275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:08:13.552991 20275 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:08:13.553076 20275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:08:13.553107 20275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:08:13.553128 20275 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:08:13.553149 20275 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:08:13.558766 20275 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:40911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:38443
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:38443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:08:13.559967 20275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:08:13.561076 20275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:08:13.569835 20284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:13.569868 20275 server_base.cc:1034] running on GCE node
W20260812 19:08:13.569824 20282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:08:13.569824 20281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:13.570725 20275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:08:13.571923 20275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:08:13.573102 20275 hybrid_clock.cc:648] HybridClock initialized: now 1786561693573089 us; error 30 us; skew 500 ppm
Aug 12 19:08:13 dist-test-slave-49h3 krb5kdc[20261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561693, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:08:13.576002 20275 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:08:13.577026 20275 webserver.cc:460] Webserver started at http://127.25.130.62:43891/ using document root <none> and password file <none>
I20260812 19:08:13.577663 20275 fs_manager.cc:359] Metadata directory not provided
I20260812 19:08:13.577744 20275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:08:13.577968 20275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:08:13.580119 20275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "40636e0403ec40039f675143cd5c0e98"
format_stamp: "Formatted at 2026-08-12 19:08:13 on dist-test-slave-49h3"
server_key: "4e38a4db585534c1473d57b6f03be7fe"
server_key_iv: "0b7575fa7f10eefdd971952280bb2b9a"
server_key_version: "encryptionkey@0"
I20260812 19:08:13.580699 20275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "40636e0403ec40039f675143cd5c0e98"
format_stamp: "Formatted at 2026-08-12 19:08:13 on dist-test-slave-49h3"
server_key: "4e38a4db585534c1473d57b6f03be7fe"
server_key_iv: "0b7575fa7f10eefdd971952280bb2b9a"
server_key_version: "encryptionkey@0"
I20260812 19:08:13.584699 20275 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 19:08:13.587302 20292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:08:13.588580 20275 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:08:13.588709 20275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "40636e0403ec40039f675143cd5c0e98"
format_stamp: "Formatted at 2026-08-12 19:08:13 on dist-test-slave-49h3"
server_key: "4e38a4db585534c1473d57b6f03be7fe"
server_key_iv: "0b7575fa7f10eefdd971952280bb2b9a"
server_key_version: "encryptionkey@0"
I20260812 19:08:13.588817 20275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:08:13.593549 20275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:08:13.596405 20275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:08:13.596587 20275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:08:13.603751 20275 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:38443
I20260812 19:08:13.603771 20353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:38443 every 8 connection(s)
I20260812 19:08:13.604750 20275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:08:13.607622 20355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:08:13.612812 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 20275
I20260812 19:08:13.612954 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:08:13.613224 26120 external_mini_cluster.cc:1428] Setting key 64128ef1727f1eeb6d177d9cda11cdd4
I20260812 19:08:13.613785 20355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40636e0403ec40039f675143cd5c0e98: Bootstrap starting.
I20260812 19:08:13.616371 20355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40636e0403ec40039f675143cd5c0e98: Neither blocks nor log segments found. Creating new log.
I20260812 19:08:13.617298 20355 log.cc:824] T 00000000000000000000000000000000 P 40636e0403ec40039f675143cd5c0e98: Log is configured to *not* fsync() on all Append() calls
I20260812 19:08:13.618995 20358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:13.614740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47521 (local address 127.25.130.62:38443)
0812 19:08:13.615249 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:13.615264 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:13.615302 (+ 38us) server_negotiation.cc:407] Connection header received
0812 19:08:13.615975 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:13.615992 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:13.616299 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:13.616627 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:13.617531 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:13.617536 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:13.617549 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:08:13.617560 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:13.617626 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:08:13.617647 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:13.617753 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:13.617883 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 19:08:13.618048 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":157,"threads_started":1}
I20260812 19:08:13.619612 20355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40636e0403ec40039f675143cd5c0e98: No bootstrap required, opened a new log
I20260812 19:08:13.622008 26120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:08:13.624245 20355 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40636e0403ec40039f675143cd5c0e98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40636e0403ec40039f675143cd5c0e98" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 38443 } }
I20260812 19:08:13.624543 20355 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40636e0403ec40039f675143cd5c0e98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:08:13.624605 20355 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40636e0403ec40039f675143cd5c0e98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40636e0403ec40039f675143cd5c0e98, State: Initialized, Role: FOLLOWER
I20260812 19:08:13.625092 20355 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40636e0403ec40039f675143cd5c0e98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40636e0403ec40039f675143cd5c0e98" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 38443 } }
I20260812 19:08:13.625249 20355 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40636e0403ec40039f675143cd5c0e98 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:08:13.625317 20355 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40636e0403ec40039f675143cd5c0e98 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:08:13.625416 20355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40636e0403ec40039f675143cd5c0e98 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:08:13.626106 20358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:13.624410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46313 (local address 127.25.130.62:38443)
0812 19:08:13.624539 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:13.624543 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:13.624789 (+ 246us) server_negotiation.cc:407] Connection header received
0812 19:08:13.624900 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:13.624903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:13.624942 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:13.625024 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:13.625739 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:13.625742 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:13.625745 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:08:13.625758 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:13.625788 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:08:13.625801 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:13.625882 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:13.625935 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 19:08:13.625974 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 19:08:13.626646 20355 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40636e0403ec40039f675143cd5c0e98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40636e0403ec40039f675143cd5c0e98" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 38443 } }
I20260812 19:08:13.627014 20355 leader_election.cc:302] T 00000000000000000000000000000000 P 40636e0403ec40039f675143cd5c0e98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40636e0403ec40039f675143cd5c0e98; no voters:
I20260812 19:08:13.627374 20355 leader_election.cc:288] T 00000000000000000000000000000000 P 40636e0403ec40039f675143cd5c0e98 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:08:13.627462 20366 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40636e0403ec40039f675143cd5c0e98 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:08:13.627642 20366 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40636e0403ec40039f675143cd5c0e98 [term 1 LEADER]: Becoming Leader. State: Replica: 40636e0403ec40039f675143cd5c0e98, State: Running, Role: LEADER
I20260812 19:08:13.627988 20366 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40636e0403ec40039f675143cd5c0e98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40636e0403ec40039f675143cd5c0e98" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 38443 } }
I20260812 19:08:13.628571 20355 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40636e0403ec40039f675143cd5c0e98 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:08:13.629598 20372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40636e0403ec40039f675143cd5c0e98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40636e0403ec40039f675143cd5c0e98. Latest consensus state: current_term: 1 leader_uuid: "40636e0403ec40039f675143cd5c0e98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40636e0403ec40039f675143cd5c0e98" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 38443 } } }
I20260812 19:08:13.629733 20372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40636e0403ec40039f675143cd5c0e98 [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:13.630126 20380 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:08:13.630194 20367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40636e0403ec40039f675143cd5c0e98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40636e0403ec40039f675143cd5c0e98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40636e0403ec40039f675143cd5c0e98" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 38443 } } }
I20260812 19:08:13.630290 20367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40636e0403ec40039f675143cd5c0e98 [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:13.633219 20380 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:08:13.638614 20380 catalog_manager.cc:1347] Generated new cluster ID: 3b2963f4bf514ffa9a5ee30053ea147a
I20260812 19:08:13.638706 20380 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:08:13.649793 20380 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:08:13.650113 20380 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:08:13.665998 20380 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40636e0403ec40039f675143cd5c0e98: Generated new TSK 0
I20260812 19:08:13.666793 20380 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:08:13.674243 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 20275
2026-08-12T19:08:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1421 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:08:13 dist-test-slave-49h3 krb5kdc[20397](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:08:13 dist-test-slave-49h3 krb5kdc[20397](info): set up 2 sockets
Aug 12 19:08:13 dist-test-slave-49h3 krb5kdc[20397](info): commencing operation
krb5kdc: starting...
W20260812 19:08:14.731578 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:08:14 dist-test-slave-49h3 krb5kdc[20397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:08:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:08:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:08:14.860426 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:42873
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:44787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:42873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:08:14.974530 20414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:08:14.974860 20414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:08:14.974937 20414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:08:14.979099 20414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:08:14.979174 20414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:08:14.979209 20414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:08:14.979240 20414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:08:14.979272 20414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:08:14.984972 20414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:44787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:42873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:42873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:08:14.986212 20414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:08:14.987262 20414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:08:14.995937 20419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:08:14.995939 20420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:08:14.995937 20422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:14.996121 20414 server_base.cc:1034] running on GCE node
I20260812 19:08:14.996867 20414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:08:14.998178 20414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:08:14.999351 20414 hybrid_clock.cc:648] HybridClock initialized: now 1786561694999333 us; error 35 us; skew 500 ppm
Aug 12 19:08:15 dist-test-slave-49h3 krb5kdc[20397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561695, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:08:15.002190 20414 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:08:15.003131 20414 webserver.cc:460] Webserver started at http://127.25.130.62:37265/ using document root <none> and password file <none>
I20260812 19:08:15.003681 20414 fs_manager.cc:359] Metadata directory not provided
I20260812 19:08:15.003744 20414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:08:15.003925 20414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:08:15.006203 20414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "fd324dd014b84f46a4722d7197d323ca"
format_stamp: "Formatted at 2026-08-12 19:08:15 on dist-test-slave-49h3"
server_key: "edc87368f5dcc2ab2384de8b467a31e2"
server_key_iv: "6fce34be0b354e01abd4f5efe95325f6"
server_key_version: "encryptionkey@0"
I20260812 19:08:15.006774 20414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "fd324dd014b84f46a4722d7197d323ca"
format_stamp: "Formatted at 2026-08-12 19:08:15 on dist-test-slave-49h3"
server_key: "edc87368f5dcc2ab2384de8b467a31e2"
server_key_iv: "6fce34be0b354e01abd4f5efe95325f6"
server_key_version: "encryptionkey@0"
I20260812 19:08:15.010718 20414 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260812 19:08:15.013276 20430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:08:15.014591 20414 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:08:15.014724 20414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "fd324dd014b84f46a4722d7197d323ca"
format_stamp: "Formatted at 2026-08-12 19:08:15 on dist-test-slave-49h3"
server_key: "edc87368f5dcc2ab2384de8b467a31e2"
server_key_iv: "6fce34be0b354e01abd4f5efe95325f6"
server_key_version: "encryptionkey@0"
I20260812 19:08:15.014842 20414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:08:15.042353 20414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:08:15.045598 20414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:08:15.045796 20414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:08:15.052757 20414 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:42873
I20260812 19:08:15.052794 20493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:42873 every 8 connection(s)
I20260812 19:08:15.053830 20414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:08:15.056680 20494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:08:15.058188 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 20414
I20260812 19:08:15.058320 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:08:15.058645 26120 external_mini_cluster.cc:1428] Setting key c7e25942dff6e88109aef4a16c501bc8
I20260812 19:08:15.063391 20494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd324dd014b84f46a4722d7197d323ca: Bootstrap starting.
I20260812 19:08:15.065747 20494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd324dd014b84f46a4722d7197d323ca: Neither blocks nor log segments found. Creating new log.
I20260812 19:08:15.066601 20494 log.cc:824] T 00000000000000000000000000000000 P fd324dd014b84f46a4722d7197d323ca: Log is configured to *not* fsync() on all Append() calls
Aug 12 19:08:15 dist-test-slave-49h3 krb5kdc[20397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:08:15.068668 20494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd324dd014b84f46a4722d7197d323ca: No bootstrap required, opened a new log
I20260812 19:08:15.073434 20497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:15.060172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57595 (local address 127.25.130.62:42873)
0812 19:08:15.060661 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:15.060677 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:15.060716 (+ 39us) server_negotiation.cc:407] Connection header received
0812 19:08:15.061368 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:15.061385 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:15.061775 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:15.062099 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:15.063015 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:15.064249 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:15.065131 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:15.065661 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:15.065811 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:15.068302 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:15.068309 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:15.068326 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:15.068361 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:15.070276 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:15.070694 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:15.070701 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:15.070708 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:15.070860 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:15.071100 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:15.071103 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:15.071105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:15.071668 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:15.071849 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:15.072058 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 19:08:15.072389 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":153,"threads_started":1}
I20260812 19:08:15.073381 20494 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd324dd014b84f46a4722d7197d323ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd324dd014b84f46a4722d7197d323ca" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42873 } }
I20260812 19:08:15.073738 20494 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd324dd014b84f46a4722d7197d323ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:08:15.073803 20494 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd324dd014b84f46a4722d7197d323ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd324dd014b84f46a4722d7197d323ca, State: Initialized, Role: FOLLOWER
I20260812 19:08:15.074270 20494 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd324dd014b84f46a4722d7197d323ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd324dd014b84f46a4722d7197d323ca" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42873 } }
I20260812 19:08:15.074409 20494 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd324dd014b84f46a4722d7197d323ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:08:15.074474 20494 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd324dd014b84f46a4722d7197d323ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:08:15.074561 20494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd324dd014b84f46a4722d7197d323ca [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:08:15.075577 20494 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd324dd014b84f46a4722d7197d323ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd324dd014b84f46a4722d7197d323ca" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42873 } }
I20260812 19:08:15.075942 20494 leader_election.cc:302] T 00000000000000000000000000000000 P fd324dd014b84f46a4722d7197d323ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd324dd014b84f46a4722d7197d323ca; no voters:
I20260812 19:08:15.076309 20494 leader_election.cc:288] T 00000000000000000000000000000000 P fd324dd014b84f46a4722d7197d323ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:08:15.076416 20499 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd324dd014b84f46a4722d7197d323ca [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:08:15.076457 26120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:08:15.076632 20499 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd324dd014b84f46a4722d7197d323ca [term 1 LEADER]: Becoming Leader. State: Replica: fd324dd014b84f46a4722d7197d323ca, State: Running, Role: LEADER
I20260812 19:08:15.076998 20499 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd324dd014b84f46a4722d7197d323ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd324dd014b84f46a4722d7197d323ca" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42873 } }
I20260812 19:08:15.077625 20494 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd324dd014b84f46a4722d7197d323ca [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:08:15.078296 20500 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd324dd014b84f46a4722d7197d323ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd324dd014b84f46a4722d7197d323ca. Latest consensus state: current_term: 1 leader_uuid: "fd324dd014b84f46a4722d7197d323ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd324dd014b84f46a4722d7197d323ca" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42873 } } }
I20260812 19:08:15.078429 20500 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd324dd014b84f46a4722d7197d323ca [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:15.078801 20511 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:08:15.079069 20501 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd324dd014b84f46a4722d7197d323ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd324dd014b84f46a4722d7197d323ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd324dd014b84f46a4722d7197d323ca" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42873 } } }
I20260812 19:08:15.079172 20501 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd324dd014b84f46a4722d7197d323ca [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:15.082074 20511 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:08:15.087723 20511 catalog_manager.cc:1347] Generated new cluster ID: 154147e1c3ce4916a37ef9326f435dee
I20260812 19:08:15.087792 20511 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:08:15.088972 20497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:15.079628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43379 (local address 127.25.130.62:42873)
0812 19:08:15.079760 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:15.079765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:15.080303 (+ 538us) server_negotiation.cc:407] Connection header received
0812 19:08:15.080481 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:15.080485 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:15.080534 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:15.080616 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:15.081401 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:15.082174 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:15.083325 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:15.083673 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:15.083789 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:15.085599 (+ 1810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:15.085603 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:15.085606 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:15.085634 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:15.087111 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:15.087563 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:15.087566 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:15.087568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:15.087664 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:15.087981 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:15.087985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:15.087987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:15.088246 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:15.088384 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:15.088665 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 19:08:15.088807 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 19:08:15.099517 20511 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:08:15.100632 20511 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:08:15.111791 20511 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd324dd014b84f46a4722d7197d323ca: Generated new TSK 0
I20260812 19:08:15.112578 20511 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:08:15.116559 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 20414
2026-08-12T19:08:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1446 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:08:15 dist-test-slave-49h3 krb5kdc[20528](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:08:15 dist-test-slave-49h3 krb5kdc[20528](info): set up 2 sockets
Aug 12 19:08:15 dist-test-slave-49h3 krb5kdc[20528](info): commencing operation
krb5kdc: starting...
W20260812 19:08:16.178539 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:08:16 dist-test-slave-49h3 krb5kdc[20528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:08:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:08:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:08:16.306629 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:40853
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:36937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:40853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:08:16.429257 20543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:08:16.429636 20543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:08:16.429680 20543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:08:16.433833 20543 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:08:16.433897 20543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:08:16.433916 20543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:08:16.433933 20543 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:08:16.433946 20543 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:08:16.439556 20543 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:36937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:40853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:40853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:08:16.440842 20543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:08:16.442085 20543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:08:16.451957 20543 server_base.cc:1034] running on GCE node
W20260812 19:08:16.451906 20550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:08:16.451894 20553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:08:16.451890 20555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:16.452822 20543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:08:16.454208 20543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:08:16.455395 20543 hybrid_clock.cc:648] HybridClock initialized: now 1786561696455368 us; error 42 us; skew 500 ppm
Aug 12 19:08:16 dist-test-slave-49h3 krb5kdc[20528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561696, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:08:16.466970 20543 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:08:16.468144 20543 webserver.cc:460] Webserver started at http://127.25.130.62:37617/ using document root <none> and password file <none>
I20260812 19:08:16.468812 20543 fs_manager.cc:359] Metadata directory not provided
I20260812 19:08:16.468878 20543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:08:16.469089 20543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:08:16.471598 20543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "14a37610b30b4d3abd0e9524306958b5"
format_stamp: "Formatted at 2026-08-12 19:08:16 on dist-test-slave-49h3"
server_key: "1a1ace35328dc5cb521a6b0ca80b64dd"
server_key_iv: "ce1e2d0efe0db57d9a4b2784613dffcb"
server_key_version: "encryptionkey@0"
I20260812 19:08:16.472167 20543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "14a37610b30b4d3abd0e9524306958b5"
format_stamp: "Formatted at 2026-08-12 19:08:16 on dist-test-slave-49h3"
server_key: "1a1ace35328dc5cb521a6b0ca80b64dd"
server_key_iv: "ce1e2d0efe0db57d9a4b2784613dffcb"
server_key_version: "encryptionkey@0"
I20260812 19:08:16.476372 20543 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 19:08:16.478979 20565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:08:16.480399 20543 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 19:08:16.480532 20543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "14a37610b30b4d3abd0e9524306958b5"
format_stamp: "Formatted at 2026-08-12 19:08:16 on dist-test-slave-49h3"
server_key: "1a1ace35328dc5cb521a6b0ca80b64dd"
server_key_iv: "ce1e2d0efe0db57d9a4b2784613dffcb"
server_key_version: "encryptionkey@0"
I20260812 19:08:16.480631 20543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:08:16.513708 20543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:08:16.517160 20543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:08:16.517364 20543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:08:16.525733 20543 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:40853
I20260812 19:08:16.525765 20627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:40853 every 8 connection(s)
I20260812 19:08:16.526916 20543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:08:16.530376 20628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:08:16.535143 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 20543
I20260812 19:08:16.535301 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:08:16.535667 26120 external_mini_cluster.cc:1428] Setting key 3030e41f18a7efe17830412682214ef7
I20260812 19:08:16.537564 20628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14a37610b30b4d3abd0e9524306958b5: Bootstrap starting.
I20260812 19:08:16.540256 20628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14a37610b30b4d3abd0e9524306958b5: Neither blocks nor log segments found. Creating new log.
I20260812 19:08:16.541110 20628 log.cc:824] T 00000000000000000000000000000000 P 14a37610b30b4d3abd0e9524306958b5: Log is configured to *not* fsync() on all Append() calls
I20260812 19:08:16.543339 20628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14a37610b30b4d3abd0e9524306958b5: No bootstrap required, opened a new log
I20260812 19:08:16.545918 20632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:16.537393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49343 (local address 127.25.130.62:40853)
0812 19:08:16.537948 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:16.537966 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:16.538011 (+ 45us) server_negotiation.cc:407] Connection header received
0812 19:08:16.538787 (+ 776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:16.538810 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:16.539213 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:16.539589 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:16.540069 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:16.541212 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:16.542390 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:16.542911 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:16.543069 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:16.543993 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:16.543999 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:16.544014 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:08:16.544468 (+ 454us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:16.544534 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:08:16.544558 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:16.544648 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:16.544790 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 19:08:16.545026 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":178,"threads_started":1}
I20260812 19:08:16.547611 20628 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14a37610b30b4d3abd0e9524306958b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14a37610b30b4d3abd0e9524306958b5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40853 } }
I20260812 19:08:16.547859 20628 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14a37610b30b4d3abd0e9524306958b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:08:16.547917 20628 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14a37610b30b4d3abd0e9524306958b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14a37610b30b4d3abd0e9524306958b5, State: Initialized, Role: FOLLOWER
I20260812 19:08:16.548492 20628 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14a37610b30b4d3abd0e9524306958b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14a37610b30b4d3abd0e9524306958b5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40853 } }
I20260812 19:08:16.548666 20628 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14a37610b30b4d3abd0e9524306958b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:08:16.548736 20628 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14a37610b30b4d3abd0e9524306958b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:08:16.548840 20628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14a37610b30b4d3abd0e9524306958b5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:08:16.549171 26120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:08:16.549885 20628 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14a37610b30b4d3abd0e9524306958b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14a37610b30b4d3abd0e9524306958b5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40853 } }
I20260812 19:08:16.550241 20628 leader_election.cc:302] T 00000000000000000000000000000000 P 14a37610b30b4d3abd0e9524306958b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14a37610b30b4d3abd0e9524306958b5; no voters:
I20260812 19:08:16.550560 20628 leader_election.cc:288] T 00000000000000000000000000000000 P 14a37610b30b4d3abd0e9524306958b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:08:16.550690 20636 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14a37610b30b4d3abd0e9524306958b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:08:16.550935 20636 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14a37610b30b4d3abd0e9524306958b5 [term 1 LEADER]: Becoming Leader. State: Replica: 14a37610b30b4d3abd0e9524306958b5, State: Running, Role: LEADER
I20260812 19:08:16.551347 20636 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14a37610b30b4d3abd0e9524306958b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14a37610b30b4d3abd0e9524306958b5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40853 } }
I20260812 19:08:16.552098 20628 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14a37610b30b4d3abd0e9524306958b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:08:16.553262 20641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14a37610b30b4d3abd0e9524306958b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14a37610b30b4d3abd0e9524306958b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14a37610b30b4d3abd0e9524306958b5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40853 } } }
I20260812 19:08:16.553416 20641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14a37610b30b4d3abd0e9524306958b5 [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:16.553728 20642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14a37610b30b4d3abd0e9524306958b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14a37610b30b4d3abd0e9524306958b5. Latest consensus state: current_term: 1 leader_uuid: "14a37610b30b4d3abd0e9524306958b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14a37610b30b4d3abd0e9524306958b5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 40853 } } }
I20260812 19:08:16.553815 20642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14a37610b30b4d3abd0e9524306958b5 [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:16.553879 20645 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:08:16.556370 20632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:16.551967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32959 (local address 127.25.130.62:40853)
0812 19:08:16.552073 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:16.552077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:16.552091 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:08:16.552291 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:16.552295 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:16.552348 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:16.552431 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:16.552755 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:16.553392 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:16.554546 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:16.554895 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:16.555000 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:16.555930 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:16.555934 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:16.555938 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:08:16.555951 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:16.555986 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:08:16.556001 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:16.556068 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:16.556169 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 19:08:16.556230 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 19:08:16.557667 20645 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:08:16.564374 20645 catalog_manager.cc:1347] Generated new cluster ID: eaa6f44f6b7c4293bbcf35c49a05d883
I20260812 19:08:16.564450 20645 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:08:16.585824 20645 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:08:16.587004 20645 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:08:16.595295 20645 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14a37610b30b4d3abd0e9524306958b5: Generated new TSK 0
I20260812 19:08:16.596158 20645 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:08:16.609488 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 20543
2026-08-12T19:08:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1490 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:08:16 dist-test-slave-49h3 krb5kdc[20666](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:08:16 dist-test-slave-49h3 krb5kdc[20666](info): set up 2 sockets
Aug 12 19:08:16 dist-test-slave-49h3 krb5kdc[20666](info): commencing operation
krb5kdc: starting...
W20260812 19:08:17.681100 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:08:17 dist-test-slave-49h3 krb5kdc[20666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:08:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:08:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:08:17.808020 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:34429
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:38829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:34429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:08:17.927316 20685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:08:17.927623 20685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:08:17.927668 20685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:08:17.931798 20685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:08:17.931875 20685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:08:17.931893 20685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:08:17.931908 20685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:08:17.931921 20685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:08:17.939057 20685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:38829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:34429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:34429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:08:17.940542 20685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:08:17.942045 20685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:08:17.951586 20691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:08:17.951577 20690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:08:17.951577 20693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:17.951855 20685 server_base.cc:1034] running on GCE node
I20260812 19:08:17.952482 20685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:08:17.953776 20685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:08:17.954943 20685 hybrid_clock.cc:648] HybridClock initialized: now 1786561697954927 us; error 33 us; skew 500 ppm
Aug 12 19:08:17 dist-test-slave-49h3 krb5kdc[20666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561697, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:08:17.957846 20685 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:08:17.958807 20685 webserver.cc:460] Webserver started at http://127.25.130.62:33143/ using document root <none> and password file <none>
I20260812 19:08:17.959406 20685 fs_manager.cc:359] Metadata directory not provided
I20260812 19:08:17.959477 20685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:08:17.959714 20685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:08:17.961982 20685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "0a49dd9c71e148ffa609ec0666e7f832"
format_stamp: "Formatted at 2026-08-12 19:08:17 on dist-test-slave-49h3"
server_key: "907c461f2cc9724cb79b9a9c92d172d1"
server_key_iv: "d835bcc47a9366eb6cc9c619463c1ad0"
server_key_version: "encryptionkey@0"
I20260812 19:08:17.962586 20685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "0a49dd9c71e148ffa609ec0666e7f832"
format_stamp: "Formatted at 2026-08-12 19:08:17 on dist-test-slave-49h3"
server_key: "907c461f2cc9724cb79b9a9c92d172d1"
server_key_iv: "d835bcc47a9366eb6cc9c619463c1ad0"
server_key_version: "encryptionkey@0"
I20260812 19:08:17.966617 20685 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 19:08:17.969316 20702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:08:17.970669 20685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:08:17.970814 20685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "0a49dd9c71e148ffa609ec0666e7f832"
format_stamp: "Formatted at 2026-08-12 19:08:17 on dist-test-slave-49h3"
server_key: "907c461f2cc9724cb79b9a9c92d172d1"
server_key_iv: "d835bcc47a9366eb6cc9c619463c1ad0"
server_key_version: "encryptionkey@0"
I20260812 19:08:17.970927 20685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:08:17.975832 20685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:08:17.979188 20685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:08:17.979374 20685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:08:17.986680 20685 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:34429
I20260812 19:08:17.986737 20764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:34429 every 8 connection(s)
I20260812 19:08:17.987707 20685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:08:17.990840 20765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:08:17.995491 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 20685
I20260812 19:08:17.995653 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:08:17.995960 26120 external_mini_cluster.cc:1428] Setting key ba566c3506e358669db1b0b6b8fb58fb
I20260812 19:08:17.997474 20765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a49dd9c71e148ffa609ec0666e7f832: Bootstrap starting.
I20260812 19:08:18.000093 20765 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a49dd9c71e148ffa609ec0666e7f832: Neither blocks nor log segments found. Creating new log.
I20260812 19:08:18.001015 20765 log.cc:824] T 00000000000000000000000000000000 P 0a49dd9c71e148ffa609ec0666e7f832: Log is configured to *not* fsync() on all Append() calls
I20260812 19:08:18.001803 20771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:17.997592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39995 (local address 127.25.130.62:34429)
0812 19:08:17.998104 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:17.998122 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:17.998162 (+ 40us) server_negotiation.cc:407] Connection header received
0812 19:08:17.998862 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:17.998879 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:17.999213 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:17.999670 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:18.000444 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:18.000449 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:18.000461 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:08:18.000472 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:18.000535 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:08:18.000556 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:18.000656 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:18.000782 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 19:08:18.000950 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":139,"threads_started":1}
I20260812 19:08:18.003253 20765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a49dd9c71e148ffa609ec0666e7f832: No bootstrap required, opened a new log
I20260812 19:08:18.004859 26120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:08:18.007544 20765 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a49dd9c71e148ffa609ec0666e7f832 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a49dd9c71e148ffa609ec0666e7f832" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 34429 } }
I20260812 19:08:18.007866 20765 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a49dd9c71e148ffa609ec0666e7f832 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:08:18.007931 20765 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a49dd9c71e148ffa609ec0666e7f832 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a49dd9c71e148ffa609ec0666e7f832, State: Initialized, Role: FOLLOWER
I20260812 19:08:18.008416 20765 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a49dd9c71e148ffa609ec0666e7f832 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a49dd9c71e148ffa609ec0666e7f832" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 34429 } }
I20260812 19:08:18.008601 20765 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a49dd9c71e148ffa609ec0666e7f832 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:08:18.008683 20765 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a49dd9c71e148ffa609ec0666e7f832 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:08:18.008739 20771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:18.007114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50719 (local address 127.25.130.62:34429)
0812 19:08:18.007234 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:18.007238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:18.007377 (+ 139us) server_negotiation.cc:407] Connection header received
0812 19:08:18.007509 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:18.007511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:18.007552 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:18.007608 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:18.008357 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:18.008360 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:18.008362 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:08:18.008372 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:18.008399 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:08:18.008409 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:18.008473 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:18.008571 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 19:08:18.008610 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 19:08:18.008795 20765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a49dd9c71e148ffa609ec0666e7f832 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:08:18.009971 20765 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a49dd9c71e148ffa609ec0666e7f832 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a49dd9c71e148ffa609ec0666e7f832" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 34429 } }
I20260812 19:08:18.010327 20765 leader_election.cc:302] T 00000000000000000000000000000000 P 0a49dd9c71e148ffa609ec0666e7f832 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a49dd9c71e148ffa609ec0666e7f832; no voters:
I20260812 19:08:18.010635 20765 leader_election.cc:288] T 00000000000000000000000000000000 P 0a49dd9c71e148ffa609ec0666e7f832 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:08:18.010737 20780 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a49dd9c71e148ffa609ec0666e7f832 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:08:18.010948 20780 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a49dd9c71e148ffa609ec0666e7f832 [term 1 LEADER]: Becoming Leader. State: Replica: 0a49dd9c71e148ffa609ec0666e7f832, State: Running, Role: LEADER
I20260812 19:08:18.011260 20780 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a49dd9c71e148ffa609ec0666e7f832 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a49dd9c71e148ffa609ec0666e7f832" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 34429 } }
I20260812 19:08:18.011888 20765 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a49dd9c71e148ffa609ec0666e7f832 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:08:18.012939 20782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a49dd9c71e148ffa609ec0666e7f832 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a49dd9c71e148ffa609ec0666e7f832. Latest consensus state: current_term: 1 leader_uuid: "0a49dd9c71e148ffa609ec0666e7f832" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a49dd9c71e148ffa609ec0666e7f832" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 34429 } } }
I20260812 19:08:18.013069 20782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a49dd9c71e148ffa609ec0666e7f832 [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:18.013362 20781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a49dd9c71e148ffa609ec0666e7f832 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a49dd9c71e148ffa609ec0666e7f832" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a49dd9c71e148ffa609ec0666e7f832" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 34429 } } }
I20260812 19:08:18.013475 20781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a49dd9c71e148ffa609ec0666e7f832 [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:18.013585 20793 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:08:18.016675 20793 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:08:18.023101 20793 catalog_manager.cc:1347] Generated new cluster ID: ff29aa12d26f4cf2b25047127b926fed
I20260812 19:08:18.023196 20793 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:08:18.040230 20793 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:08:18.040642 20793 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:08:18.046090 20793 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a49dd9c71e148ffa609ec0666e7f832: Generated new TSK 0
I20260812 19:08:18.046833 20793 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:08:18.058097 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 20685
2026-08-12T19:08:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1448 ms)
[----------] 18 tests from AuthTokenIssuingTest (25945 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:08:18 dist-test-slave-49h3 krb5kdc[20808](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:08:18 dist-test-slave-49h3 krb5kdc[20808](info): set up 2 sockets
Aug 12 19:08:18 dist-test-slave-49h3 krb5kdc[20808](info): commencing operation
krb5kdc: starting...
W20260812 19:08:19.135720 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:08:19 dist-test-slave-49h3 krb5kdc[20808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:08:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:08:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:08:19.278252 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:35673
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:46797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:35673,127.25.130.61:35211,127.25.130.60:40519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:08:19.407058 20825 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 19:08:19.407461 20825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:08:19.407500 20825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:08:19.407570 20825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:08:19.411888 20825 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:08:19.411981 20825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:08:19.412010 20825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:08:19.412032 20825 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:08:19.412057 20825 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:08:19.418155 20825 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:46797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:35673,127.25.130.61:35211,127.25.130.60:40519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:35673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:08:19.419694 20825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:08:19.421046 20825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:08:19.431389 20832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:19.431392 20825 server_base.cc:1034] running on GCE node
W20260812 19:08:19.431456 20831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:08:19.431442 20834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:19.432353 20825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:08:19.433785 20825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:08:19.434963 20825 hybrid_clock.cc:648] HybridClock initialized: now 1786561699434929 us; error 43 us; skew 500 ppm
Aug 12 19:08:19 dist-test-slave-49h3 krb5kdc[20808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561699, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:08:19.438192 20825 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:08:19.439301 20825 webserver.cc:460] Webserver started at http://127.25.130.62:38905/ using document root <none> and password file <none>
I20260812 19:08:19.439976 20825 fs_manager.cc:359] Metadata directory not provided
I20260812 19:08:19.440047 20825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:08:19.440303 20825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:08:19.442978 20825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "7fb91cae50d4433f854e33fc362c3ecb"
format_stamp: "Formatted at 2026-08-12 19:08:19 on dist-test-slave-49h3"
server_key: "1a7dbb2bb000a1a56feb2b2981f30326"
server_key_iv: "83591a9e2caf09980b4d8c41d0a69781"
server_key_version: "encryptionkey@0"
I20260812 19:08:19.443653 20825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "7fb91cae50d4433f854e33fc362c3ecb"
format_stamp: "Formatted at 2026-08-12 19:08:19 on dist-test-slave-49h3"
server_key: "1a7dbb2bb000a1a56feb2b2981f30326"
server_key_iv: "83591a9e2caf09980b4d8c41d0a69781"
server_key_version: "encryptionkey@0"
I20260812 19:08:19.448990 20825 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 19:08:19.452427 20845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:08:19.454378 20825 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 19:08:19.454535 20825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "7fb91cae50d4433f854e33fc362c3ecb"
format_stamp: "Formatted at 2026-08-12 19:08:19 on dist-test-slave-49h3"
server_key: "1a7dbb2bb000a1a56feb2b2981f30326"
server_key_iv: "83591a9e2caf09980b4d8c41d0a69781"
server_key_version: "encryptionkey@0"
I20260812 19:08:19.454711 20825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:08:19.468654 20825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:08:19.472177 20825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:08:19.472355 20825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:08:19.479763 20825 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:35673
I20260812 19:08:19.480087 20911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:35673 every 8 connection(s)
I20260812 19:08:19.480849 20825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:08:19.483891 20912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:08:19.487399 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 20825
I20260812 19:08:19.487569 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:08:19.487887 26120 external_mini_cluster.cc:1428] Setting key 305791019a2a8b8f45c10103abd9290c
I20260812 19:08:19.488421 20912 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.62" port: 35673 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.25.130.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.61@KRBTEST.COM" created.
Aug 12 19:08:19 dist-test-slave-49h3 krb5kdc[20808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561699, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:08:19.505378 20915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.491367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41818 (local address 127.25.130.62:35673)
0812 19:08:19.491854 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:19.491878 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:19.492135 (+ 257us) server_negotiation.cc:407] Connection header received
0812 19:08:19.493221 (+ 1086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:19.493226 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:19.493468 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:19.493653 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:19.494529 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.495427 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.496392 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.496764 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.496909 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:19.500624 (+ 3715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:19.500631 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:19.500646 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:19.500673 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:19.502305 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:19.502857 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:19.502862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:19.502863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:19.502969 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:19.503295 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:19.503298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:19.503299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:19.503700 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:19.503803 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:19.504275 (+ 472us) server_negotiation.cc:299] Negotiation successful
0812 19:08:19.504512 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":336,"thread_start_us":157,"threads_started":1}
I20260812 19:08:19.505384 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.491368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:35673 (local address 127.0.0.1:41818)
0812 19:08:19.491988 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:19.492026 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 19:08:19.492911 (+ 885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:08:19.493663 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:08:19.493668 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:08:19.494127 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:08:19.494380 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:19.494393 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.495551 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.495554 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:19.496281 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:19.496290 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.496913 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.496917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:19.497057 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:19.497596 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:08:19.497613 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:08:19.500455 (+ 2842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:08:19.502418 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:19.502428 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:19.502440 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:19.502754 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:19.503054 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:19.503057 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:19.503059 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:19.503202 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:19.503820 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:08:19.503828 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:08:19.504000 (+ 172us) client_negotiation.cc:769] Sending connection context
0812 19:08:19.504192 (+ 192us) client_negotiation.cc:240] Negotiation successful
0812 19:08:19.504613 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":86,"spinlock_wait_cycles":46464,"thread_start_us":186,"threads_started":1}
I20260812 19:08:19.507956 20912 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.61" port: 35211 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:08:19.508710 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.508443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:35211 (local address 127.0.0.1:57392)
0812 19:08:19.508575 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:19.508647 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.61:35211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 19:08:19.508889 20848 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.130.61:35211: connect: Connection refused (error 111)
I20260812 19:08:19.509814 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.509580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:35211 (local address 127.0.0.1:57408)
0812 19:08:19.509682 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:19.509736 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.61:35211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 19:08:19.509945 20912 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.130.61:35211: Network error: Client connection negotiation failed: client connection to 127.25.130.61:35211: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.61.keytab.
Entry for principal kudu/127.25.130.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.61.keytab.
I20260812 19:08:19.516245 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.61:35211
--webserver_interface=127.25.130.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.61.keytab
--principal=kudu/127.25.130.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:46797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:35673,127.25.130.61:35211,127.25.130.60:40519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:08:19.559173 20912 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.130.61" port: 35211 } attempt: 1
I20260812 19:08:19.560043 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.559738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:35211 (local address 127.0.0.1:57422)
0812 19:08:19.559899 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:19.559961 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.61:35211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 19:08:19.560734 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.560518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:35211 (local address 127.0.0.1:57434)
0812 19:08:19.560622 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:19.560668 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.61:35211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 19:08:19.560851 20912 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.130.61:35211: Network error: Client connection negotiation failed: client connection to 127.25.130.61:35211: connect: Connection refused (error 111)
I20260812 19:08:19.629024 20912 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.130.61" port: 35211 } attempt: 2
I20260812 19:08:19.629928 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.629639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:35211 (local address 127.0.0.1:57438)
0812 19:08:19.629786 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:19.629849 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.61:35211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 19:08:19.630635 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.630420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:35211 (local address 127.0.0.1:57444)
0812 19:08:19.630526 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:19.630572 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.61:35211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 19:08:19.630760 20912 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.130.61:35211: Network error: Client connection negotiation failed: client connection to 127.25.130.61:35211: connect: Connection refused (error 111)
W20260812 19:08:19.639294 20919 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 19:08:19.639640 20919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:08:19.639686 20919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:08:19.639753 20919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:08:19.643908 20919 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:08:19.643976 20919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:08:19.644003 20919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:08:19.644026 20919 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:08:19.644054 20919 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:08:19.649791 20919 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:46797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:35673,127.25.130.61:35211,127.25.130.60:40519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.61.keytab
--principal=kudu/127.25.130.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.61:35211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.25.130.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:08:19.651012 20919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:08:19.652140 20919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:08:19.660953 20925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:08:19.660964 20926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:19.661010 20919 server_base.cc:1034] running on GCE node
W20260812 19:08:19.660972 20928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:19.661783 20919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:08:19.663018 20919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:08:19.664184 20919 hybrid_clock.cc:648] HybridClock initialized: now 1786561699664165 us; error 39 us; skew 500 ppm
Aug 12 19:08:19 dist-test-slave-49h3 krb5kdc[20808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561699, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:08:19.667366 20919 init.cc:377] Logged in from keytab as kudu/127.25.130.61@KRBTEST.COM (short username kudu)
I20260812 19:08:19.668298 20919 webserver.cc:460] Webserver started at http://127.25.130.61:34057/ using document root <none> and password file <none>
I20260812 19:08:19.668876 20919 fs_manager.cc:359] Metadata directory not provided
I20260812 19:08:19.668941 20919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:08:19.669162 20919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:08:19.671358 20919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-1/data/instance:
uuid: "1f526cbb4af544f6b30cda3532ee16d1"
format_stamp: "Formatted at 2026-08-12 19:08:19 on dist-test-slave-49h3"
server_key: "1207b3d330e9a94ba1d435821aa6fdb9"
server_key_iv: "4699b6c5146387a11fee3022bc0a4968"
server_key_version: "encryptionkey@0"
I20260812 19:08:19.671954 20919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-1/wal/instance:
uuid: "1f526cbb4af544f6b30cda3532ee16d1"
format_stamp: "Formatted at 2026-08-12 19:08:19 on dist-test-slave-49h3"
server_key: "1207b3d330e9a94ba1d435821aa6fdb9"
server_key_iv: "4699b6c5146387a11fee3022bc0a4968"
server_key_version: "encryptionkey@0"
I20260812 19:08:19.675854 20919 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260812 19:08:19.678232 20935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:08:19.679440 20919 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 19:08:19.679558 20919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-1/wal
uuid: "1f526cbb4af544f6b30cda3532ee16d1"
format_stamp: "Formatted at 2026-08-12 19:08:19 on dist-test-slave-49h3"
server_key: "1207b3d330e9a94ba1d435821aa6fdb9"
server_key_iv: "4699b6c5146387a11fee3022bc0a4968"
server_key_version: "encryptionkey@0"
I20260812 19:08:19.679663 20919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:08:19.703174 20919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:08:19.706209 20919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:08:19.706405 20919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:08:19.713544 20919 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.61:35211
I20260812 19:08:19.713853 20997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.61:35211 every 8 connection(s)
I20260812 19:08:19.714545 20919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-1/data/info.pb
I20260812 19:08:19.717324 20998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:08:19.721171 20998 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.62" port: 35673 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:08:19.721917 20912 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.130.61" port: 35211 } attempt: 3
I20260812 19:08:19.723903 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 20919
I20260812 19:08:19.724032 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-1/wal/instance
I20260812 19:08:19.724324 26120 external_mini_cluster.cc:1428] Setting key 382d99f91ac383618bfe1fa8308cd793
Aug 12 19:08:19 dist-test-slave-49h3 krb5kdc[20808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561699, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for kudu/127.25.130.61@KRBTEST.COM
Aug 12 19:08:19 dist-test-slave-49h3 krb5kdc[20808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561699, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.61@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.130.60@KRBTEST.COM; defaulting to no policy
I20260812 19:08:19.734732 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.722430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:35211 (local address 127.0.0.1:57454)
0812 19:08:19.722584 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:19.722600 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:08:19.722688 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:08:19.725169 (+ 2481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:08:19.725172 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:08:19.725367 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:08:19.725523 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:19.725532 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.727088 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.727091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:19.727841 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:19.727854 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.728578 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.728582 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:19.728682 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:19.729131 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:08:19.729143 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:08:19.730453 (+ 1310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:08:19.732864 (+ 2411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:19.732868 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:19.732870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:19.733110 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:19.733546 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:19.733549 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:19.733551 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:19.733630 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:19.734273 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:08:19.734277 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:08:19.734383 (+ 106us) client_negotiation.cc:769] Sending connection context
0812 19:08:19.734468 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 19:08:19.734588 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 19:08:19.735749 20915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.723111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41834 (local address 127.25.130.62:35673)
0812 19:08:19.723245 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:19.723250 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:19.723766 (+ 516us) server_negotiation.cc:407] Connection header received
0812 19:08:19.724871 (+ 1105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:19.724875 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:19.724928 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:19.724995 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:19.725975 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.726581 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.727705 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.727979 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.728090 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:19.731262 (+ 3172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:19.731265 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:19.731267 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:19.731289 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:19.732603 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:19.733049 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:19.733053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:19.733055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:19.733133 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:19.733475 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:19.733479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:19.733481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:19.733739 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:19.733822 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:19.735471 (+ 1649us) server_negotiation.cc:299] Negotiation successful
0812 19:08:19.735611 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49,"spinlock_wait_cycles":1280}
I20260812 19:08:19.735744 21000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.723095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57454 (local address 127.25.130.61:35211)
0812 19:08:19.723529 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:19.723544 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:19.723578 (+ 34us) server_negotiation.cc:407] Connection header received
0812 19:08:19.724664 (+ 1086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:19.724690 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:19.724939 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:19.725192 (+ 253us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:19.725653 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.726972 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.727953 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.728464 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.728564 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:19.730628 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:19.730637 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:19.730651 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:19.730680 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:19.732729 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:19.733256 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:19.733263 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:19.733265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:19.733377 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:19.733729 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:19.733732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:19.733734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:19.734137 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:19.734957 (+ 820us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:19.735049 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 19:08:19.735187 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":300,"thread_start_us":138,"threads_started":1}
I20260812 19:08:19.735750 21001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.723095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:35673 (local address 127.0.0.1:41834)
0812 19:08:19.723627 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:19.723678 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 19:08:19.724670 (+ 992us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:08:19.725065 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:08:19.725075 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:08:19.725588 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:08:19.725848 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:19.725859 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.726696 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.726699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:19.727584 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:19.727594 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.728146 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.728150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:19.728412 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:19.729032 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:08:19.729052 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:08:19.731101 (+ 2049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:08:19.732710 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:19.732720 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:19.732731 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:19.732957 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:19.733216 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:19.733220 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:19.733222 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:19.733365 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:19.733839 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:08:19.733846 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:08:19.734130 (+ 284us) client_negotiation.cc:769] Sending connection context
0812 19:08:19.734304 (+ 174us) client_negotiation.cc:240] Negotiation successful
0812 19:08:19.734668 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":35,"spinlock_wait_cycles":24960,"thread_start_us":182,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.60@KRBTEST.COM" created.
I20260812 19:08:19.737759 20912 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:08:19.738197 20998 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.61" port: 35211 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:08:19.738457 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.738233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:40519 (local address 127.0.0.1:35898)
0812 19:08:19.738347 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:19.738394 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:40519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 19:08:19.739391 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.739096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:40519 (local address 127.0.0.1:35910)
0812 19:08:19.739284 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:19.739331 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:40519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260812 19:08:19.739530 20912 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.130.60:40519: Network error: Client connection negotiation failed: client connection to 127.25.130.60:40519: connect: Connection refused (error 111)
Aug 12 19:08:19 dist-test-slave-49h3 krb5kdc[20808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561699, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.61@KRBTEST.COM for kudu/127.25.130.61@KRBTEST.COM
I20260812 19:08:19.746757 21000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.738930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57460 (local address 127.25.130.61:35211)
0812 19:08:19.739011 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:19.739015 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:19.739025 (+ 10us) server_negotiation.cc:407] Connection header received
0812 19:08:19.739050 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:19.739053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:19.739101 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:19.739163 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:19.739672 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.740424 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.741358 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.741632 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.741708 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:19.743821 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:19.743824 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:19.743826 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:19.743847 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:19.745073 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:19.745577 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:19.745579 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:19.745580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:19.745650 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:19.745907 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:19.745909 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:19.745910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:19.746081 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:19.746159 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:19.746421 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 19:08:19.746610 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":84,"server-negotiator.queue_time_us":26}
I20260812 19:08:19.746757 21001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.738668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:35211 (local address 127.0.0.1:57460)
0812 19:08:19.738785 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:19.738799 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:08:19.738876 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:08:19.739259 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:08:19.739262 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:08:19.739438 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:08:19.739549 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:19.739557 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.740538 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.740541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:19.741255 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:19.741266 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.741732 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.741735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:19.741830 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:19.742284 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:08:19.742299 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:08:19.743690 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:08:19.745189 (+ 1499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:19.745192 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:19.745194 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:19.745492 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:19.745732 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:19.745735 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:19.745737 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:19.745829 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:19.746173 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:08:19.746176 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:08:19.746275 (+ 99us) client_negotiation.cc:769] Sending connection context
0812 19:08:19.746392 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 19:08:19.746562 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":3}
I20260812 19:08:19.747503 20998 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:08:19.748172 21001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.747942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:40519 (local address 127.0.0.1:35924)
0812 19:08:19.748037 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:19.748119 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:40519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 19:08:19.748366 20939 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.130.60:40519: connect: Connection refused (error 111)
I20260812 19:08:19.749680 21001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.749483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:40519 (local address 127.0.0.1:35934)
0812 19:08:19.749585 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:19.749630 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:40519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 19:08:19.749821 20998 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.130.60:40519: Network error: Client connection negotiation failed: client connection to 127.25.130.60:40519: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.60.keytab.
Entry for principal kudu/127.25.130.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.60.keytab.
I20260812 19:08:19.754247 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.60:40519
--webserver_interface=127.25.130.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.60.keytab
--principal=kudu/127.25.130.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:46797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:35673,127.25.130.61:35211,127.25.130.60:40519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:08:19.770711 20912 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 } attempt: 1
I20260812 19:08:19.771605 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.771293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:40519 (local address 127.0.0.1:35936)
0812 19:08:19.771449 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:19.771512 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:40519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 19:08:19.772303 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.772089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:40519 (local address 127.0.0.1:35946)
0812 19:08:19.772196 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:19.772242 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:40519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 19:08:19.772424 20912 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.130.60:40519: Network error: Client connection negotiation failed: client connection to 127.25.130.60:40519: connect: Connection refused (error 111)
I20260812 19:08:19.799108 20998 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 } attempt: 1
I20260812 19:08:19.800043 21001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.799739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:40519 (local address 127.0.0.1:35948)
0812 19:08:19.799894 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:19.799957 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:40519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 19:08:19.800786 21001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.800592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:40519 (local address 127.0.0.1:35956)
0812 19:08:19.800682 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:19.800728 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:40519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 19:08:19.800935 20998 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.130.60:40519: Network error: Client connection negotiation failed: client connection to 127.25.130.60:40519: connect: Connection refused (error 111)
I20260812 19:08:19.847550 20912 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 } attempt: 2
I20260812 19:08:19.848454 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.848108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:40519 (local address 127.0.0.1:35966)
0812 19:08:19.848282 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:19.848359 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:40519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 19:08:19.849254 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.849002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:40519 (local address 127.0.0.1:35970)
0812 19:08:19.849128 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:19.849186 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:40519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 19:08:19.849385 20912 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.130.60:40519: Network error: Client connection negotiation failed: client connection to 127.25.130.60:40519: connect: Connection refused (error 111)
I20260812 19:08:19.869086 20998 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 } attempt: 2
W20260812 19:08:19.869148 21005 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 19:08:19.869524 21005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:08:19.869588 21005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:08:19.869649 21005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 19:08:19.869987 21001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.869691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:40519 (local address 127.0.0.1:35974)
0812 19:08:19.869839 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:19.869905 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:40519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 19:08:19.870704 21001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.870491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:40519 (local address 127.0.0.1:35986)
0812 19:08:19.870596 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:19.870646 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:40519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 19:08:19.870813 20998 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.130.60:40519: Network error: Client connection negotiation failed: client connection to 127.25.130.60:40519: connect: Connection refused (error 111)
W20260812 19:08:19.873924 21005 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:08:19.873983 21005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:08:19.874010 21005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:08:19.874040 21005 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:08:19.874064 21005 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:08:19.879550 21005 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:46797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:35673,127.25.130.61:35211,127.25.130.60:40519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.60.keytab
--principal=kudu/127.25.130.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.60:40519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.25.130.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:08:19.880666 21005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:08:19.881754 21005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:08:19.890805 21013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:19.890861 21005 server_base.cc:1034] running on GCE node
W20260812 19:08:19.890805 21015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:08:19.891057 21018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:19.891628 21005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:08:19.892786 21005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:08:19.893961 21005 hybrid_clock.cc:648] HybridClock initialized: now 1786561699893946 us; error 32 us; skew 500 ppm
Aug 12 19:08:19 dist-test-slave-49h3 krb5kdc[20808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561699, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:08:19.896704 21005 init.cc:377] Logged in from keytab as kudu/127.25.130.60@KRBTEST.COM (short username kudu)
I20260812 19:08:19.897665 21005 webserver.cc:460] Webserver started at http://127.25.130.60:43083/ using document root <none> and password file <none>
I20260812 19:08:19.898204 21005 fs_manager.cc:359] Metadata directory not provided
I20260812 19:08:19.898263 21005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:08:19.898437 21005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:08:19.900537 21005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-2/data/instance:
uuid: "041f6f0f9dc24fe28a299fdb2cc18ef7"
format_stamp: "Formatted at 2026-08-12 19:08:19 on dist-test-slave-49h3"
server_key: "3819ab3bdbbe10c4cd41f676d299a0a0"
server_key_iv: "b0a28de24795c3009ba61bece532db69"
server_key_version: "encryptionkey@0"
I20260812 19:08:19.901109 21005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-2/wal/instance:
uuid: "041f6f0f9dc24fe28a299fdb2cc18ef7"
format_stamp: "Formatted at 2026-08-12 19:08:19 on dist-test-slave-49h3"
server_key: "3819ab3bdbbe10c4cd41f676d299a0a0"
server_key_iv: "b0a28de24795c3009ba61bece532db69"
server_key_version: "encryptionkey@0"
I20260812 19:08:19.905064 21005 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 19:08:19.907588 21026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:08:19.908851 21005 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:08:19.908981 21005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-2/wal
uuid: "041f6f0f9dc24fe28a299fdb2cc18ef7"
format_stamp: "Formatted at 2026-08-12 19:08:19 on dist-test-slave-49h3"
server_key: "3819ab3bdbbe10c4cd41f676d299a0a0"
server_key_iv: "b0a28de24795c3009ba61bece532db69"
server_key_version: "encryptionkey@0"
I20260812 19:08:19.909102 21005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:08:19.945847 21005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:08:19.948541 20912 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 } attempt: 3
I20260812 19:08:19.949280 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.949006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:40519 (local address 127.0.0.1:36002)
0812 19:08:19.949138 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:19.949201 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:40519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 19:08:19.949302 21005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:08:19.949518 21005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:08:19.950011 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.949804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:40519 (local address 127.0.0.1:36010)
0812 19:08:19.949901 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:19.949949 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:40519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 19:08:19.950145 20912 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.130.60:40519: Network error: Client connection negotiation failed: client connection to 127.25.130.60:40519: connect: Connection refused (error 111)
I20260812 19:08:19.956391 21005 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.60:40519
I20260812 19:08:19.956638 21088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.60:40519 every 8 connection(s)
I20260812 19:08:19.957377 21005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-2/data/info.pb
I20260812 19:08:19.960273 21089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:08:19.961957 20998 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 } attempt: 3
I20260812 19:08:19.963259 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 21005
I20260812 19:08:19.963397 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561415988404-26120-0/minicluster-data/master-2/wal/instance
I20260812 19:08:19.963706 26120 external_mini_cluster.cc:1428] Setting key 12338111f1943aeee76bdc5cf8b38a8a
I20260812 19:08:19.964828 21089 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.62" port: 35673 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 19:08:19 dist-test-slave-49h3 krb5kdc[20808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561699, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.61@KRBTEST.COM for kudu/127.25.130.60@KRBTEST.COM
Aug 12 19:08:19 dist-test-slave-49h3 krb5kdc[20808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.60@KRBTEST.COM
Aug 12 19:08:19 dist-test-slave-49h3 krb5kdc[20808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561699, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.60@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:08:19.975554 21001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.962508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:40519 (local address 127.0.0.1:36014)
0812 19:08:19.962678 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:19.962695 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 19:08:19.962796 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:08:19.965286 (+ 2490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:08:19.965289 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:08:19.965537 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:08:19.965658 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:19.965666 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.967427 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.967431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:19.968083 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:19.968095 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.968951 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.968955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:19.969042 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:19.969628 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:08:19.969641 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:08:19.971401 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:08:19.973527 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:19.973531 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:19.973534 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:19.973783 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:19.974142 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:19.974145 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:19.974148 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:19.974247 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:19.974974 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:08:19.974978 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:08:19.975095 (+ 117us) client_negotiation.cc:769] Sending connection context
0812 19:08:19.975221 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 19:08:19.975372 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 19:08:19.976801 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.963220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36014 (local address 127.25.130.60:40519)
0812 19:08:19.963672 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:19.963688 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:19.963726 (+ 38us) server_negotiation.cc:407] Connection header received
0812 19:08:19.964526 (+ 800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:19.964553 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:19.965010 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:19.965320 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:19.965872 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.967252 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.968202 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.968816 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.968971 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:19.971577 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:19.971584 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:19.971599 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:19.971627 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:19.973390 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:19.973894 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:19.973901 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:19.973907 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:19.974047 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:19.974341 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:19.974344 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:19.974345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:19.974865 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:19.974997 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:19.975383 (+ 386us) server_negotiation.cc:299] Negotiation successful
0812 19:08:19.975710 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":302,"spinlock_wait_cycles":4480,"thread_start_us":153,"threads_started":1}
I20260812 19:08:19.976801 21093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.964369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36030 (local address 127.25.130.60:40519)
0812 19:08:19.964695 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:19.964704 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:19.964885 (+ 181us) server_negotiation.cc:407] Connection header received
0812 19:08:19.965007 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:19.965010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:19.965056 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:19.965322 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:19.966384 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.967619 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.968696 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.968943 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.969032 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:19.971855 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:19.971860 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:19.971862 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:19.971894 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:19.974130 (+ 2236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:19.974573 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:19.974576 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:19.974578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:19.974659 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:19.974976 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:19.974979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:19.974981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:19.975166 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:19.975316 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:19.975957 (+ 641us) server_negotiation.cc:299] Negotiation successful
0812 19:08:19.976082 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":878,"server-negotiator.queue_time_us":264,"spinlock_wait_cycles":15360,"thread_start_us":133,"threads_started":1}
I20260812 19:08:19.977051 20915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.966904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41838 (local address 127.25.130.62:35673)
0812 19:08:19.967041 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:19.967045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:19.967349 (+ 304us) server_negotiation.cc:407] Connection header received
0812 19:08:19.967528 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:19.967532 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:19.967587 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:19.967670 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:19.968495 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.969195 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.970241 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.970467 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.970566 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:19.973069 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:19.973073 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:19.973076 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:19.973107 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:19.974924 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:19.975466 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:19.975470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:19.975472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:19.975557 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:19.976149 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:19.976152 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:19.976154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:19.976354 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:19.976443 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:19.976775 (+ 332us) server_negotiation.cc:299] Negotiation successful
0812 19:08:19.976884 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 19:08:19.977557 21095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.966830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:35673 (local address 127.0.0.1:41838)
0812 19:08:19.967211 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:19.967264 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 19:08:19.967389 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:08:19.967735 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:08:19.967740 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:08:19.968214 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:08:19.968360 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:19.968367 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.969324 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.969327 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:19.970117 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:19.970128 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.970616 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.970620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:19.970728 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:19.971173 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:08:19.971193 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:08:19.972904 (+ 1711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:08:19.975044 (+ 2140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:19.975055 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:19.975060 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:19.975352 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:19.975947 (+ 595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:19.975949 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:19.975951 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:19.976042 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:19.976460 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:08:19.976467 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:08:19.976574 (+ 107us) client_negotiation.cc:769] Sending connection context
0812 19:08:19.976700 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 19:08:19.976883 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"spinlock_wait_cycles":2304,"thread_start_us":110,"threads_started":1}
I20260812 19:08:19.978775 21089 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.61" port: 35211 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:08:19.981223 20998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f526cbb4af544f6b30cda3532ee16d1: Bootstrap starting.
Aug 12 19:08:19 dist-test-slave-49h3 krb5kdc[20808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561699, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.60@KRBTEST.COM for kudu/127.25.130.61@KRBTEST.COM
I20260812 19:08:19.984272 20998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f526cbb4af544f6b30cda3532ee16d1: Neither blocks nor log segments found. Creating new log.
I20260812 19:08:19.985153 20998 log.cc:824] T 00000000000000000000000000000000 P 1f526cbb4af544f6b30cda3532ee16d1: Log is configured to *not* fsync() on all Append() calls
Aug 12 19:08:19 dist-test-slave-49h3 krb5kdc[20808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:08:19.987498 20998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f526cbb4af544f6b30cda3532ee16d1: No bootstrap required, opened a new log
I20260812 19:08:19.987823 21095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.979387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:35211 (local address 127.0.0.1:57470)
0812 19:08:19.979496 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:19.979519 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 19:08:19.979599 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:08:19.979816 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:08:19.979819 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:08:19.979976 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:08:19.980072 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:19.980077 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.980769 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.980773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:19.981274 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:19.981281 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.981810 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.981814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:19.981908 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:19.982335 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:08:19.982347 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:08:19.983827 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:08:19.985948 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:19.985952 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:19.985954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:19.986225 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:19.986693 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:19.986696 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:19.986698 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:19.986844 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:19.987322 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:08:19.987325 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:08:19.987419 (+ 94us) client_negotiation.cc:769] Sending connection context
0812 19:08:19.987525 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 19:08:19.987672 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 19:08:19.988090 21000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.979454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57470 (local address 127.25.130.61:35211)
0812 19:08:19.979563 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:19.979568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:19.979580 (+ 12us) server_negotiation.cc:407] Connection header received
0812 19:08:19.979682 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:19.979685 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:19.979727 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:19.979791 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:19.980178 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.980670 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.981377 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.981699 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.981796 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:19.983955 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:19.983959 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:19.983962 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:19.983988 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:19.985824 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:19.986342 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:19.986345 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:19.986347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:19.986526 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:19.986959 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:19.986963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:19.986965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:19.987224 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:19.987313 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:19.987674 (+ 361us) server_negotiation.cc:299] Negotiation successful
0812 19:08:19.987793 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 19:08:19.988696 21089 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:08:19.991459 20915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.981581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41846 (local address 127.25.130.62:35673)
0812 19:08:19.981719 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:19.981723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:19.981737 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:08:19.981775 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:19.981777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:19.981819 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:19.981891 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:19.982780 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.983307 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.984266 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.984585 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.984692 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:19.987132 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:19.987137 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:19.987139 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:19.987177 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:19.989676 (+ 2499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:19.990193 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:19.990198 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:19.990199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:19.990290 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:19.990584 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:19.990587 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:19.990589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:19.990808 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:19.991032 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:19.991191 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 19:08:19.991314 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 19:08:19.991775 20998 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f526cbb4af544f6b30cda3532ee16d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb91cae50d4433f854e33fc362c3ecb" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 35673 } } peers { permanent_uuid: "1f526cbb4af544f6b30cda3532ee16d1" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 35211 } } peers { permanent_uuid: "041f6f0f9dc24fe28a299fdb2cc18ef7" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 } }
I20260812 19:08:19.992023 20998 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f526cbb4af544f6b30cda3532ee16d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f526cbb4af544f6b30cda3532ee16d1, State: Initialized, Role: FOLLOWER
I20260812 19:08:19.992388 20998 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f526cbb4af544f6b30cda3532ee16d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb91cae50d4433f854e33fc362c3ecb" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 35673 } } peers { permanent_uuid: "1f526cbb4af544f6b30cda3532ee16d1" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 35211 } } peers { permanent_uuid: "041f6f0f9dc24fe28a299fdb2cc18ef7" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 } }
Aug 12 19:08:19 dist-test-slave-49h3 krb5kdc[20808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561699, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.60@KRBTEST.COM for kudu/127.25.130.60@KRBTEST.COM
I20260812 19:08:19.993747 21098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f526cbb4af544f6b30cda3532ee16d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb91cae50d4433f854e33fc362c3ecb" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 35673 } } peers { permanent_uuid: "1f526cbb4af544f6b30cda3532ee16d1" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 35211 } } peers { permanent_uuid: "041f6f0f9dc24fe28a299fdb2cc18ef7" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 } } }
I20260812 19:08:19.993911 21098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f526cbb4af544f6b30cda3532ee16d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:08:19.994055 20998 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f526cbb4af544f6b30cda3532ee16d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:08:19.998201 21095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.989243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:40519 (local address 127.0.0.1:36034)
0812 19:08:19.989404 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:19.989417 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:08:19.989541 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:08:19.989728 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:08:19.989731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:08:19.989882 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:08:19.989947 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:19.989952 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.990637 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.990640 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:19.991269 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:19.991280 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.992089 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.992094 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:19.992184 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:19.992614 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:08:19.992629 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:08:19.994077 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:08:19.996523 (+ 2446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:19.996526 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:19.996528 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:19.996777 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:19.997100 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:19.997102 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:19.997104 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:19.997190 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:19.997709 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:08:19.997711 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:08:19.997797 (+ 86us) client_negotiation.cc:769] Sending connection context
0812 19:08:19.997886 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 19:08:19.998044 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111}
Aug 12 19:08:19 dist-test-slave-49h3 krb5kdc[20808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.61@KRBTEST.COM
I20260812 19:08:19.998647 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.989273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36034 (local address 127.25.130.60:40519)
0812 19:08:19.989367 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:19.989371 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:19.989516 (+ 145us) server_negotiation.cc:407] Connection header received
0812 19:08:19.989608 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:19.989611 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:19.989646 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:19.989709 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:19.990049 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.990543 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.991616 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.991917 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.992062 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:19.994226 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:19.994229 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:19.994232 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:19.994256 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:19.995931 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:19.996886 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:19.996890 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:19.996892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:19.996989 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:19.997276 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:19.997279 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:19.997280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:19.997578 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:19.997675 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:19.998410 (+ 735us) server_negotiation.cc:299] Negotiation successful
0812 19:08:19.998520 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260812 19:08:20.000527 21114 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1f526cbb4af544f6b30cda3532ee16d1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:08:20.000666 21114 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:08:19.998199 20919 master_runner.cc:186] Error getting master registration for 127.25.130.62:35673: OK, instance_id { permanent_uuid: "7fb91cae50d4433f854e33fc362c3ecb" instance_seqno: 1786561699477774 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 19:08:20.001506 20919 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 19:08:20.002060 21089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 041f6f0f9dc24fe28a299fdb2cc18ef7: Bootstrap starting.
I20260812 19:08:20.002525 21000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:19.992797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57472 (local address 127.25.130.61:35211)
0812 19:08:19.992914 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:19.992918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:19.992933 (+ 15us) server_negotiation.cc:407] Connection header received
0812 19:08:19.993050 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:19.993053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:19.993103 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:19.993175 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:19.994056 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.994572 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.995566 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:19.995886 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:19.995993 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:19.999225 (+ 3232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:19.999230 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:19.999232 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:19.999259 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:20.000864 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:20.001262 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:20.001266 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:20.001268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:20.001350 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:20.001663 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:20.001667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:20.001668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:20.001843 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:20.002005 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:20.002291 (+ 286us) server_negotiation.cc:299] Negotiation successful
0812 19:08:20.002399 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 19:08:20.004864 21089 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 041f6f0f9dc24fe28a299fdb2cc18ef7: Neither blocks nor log segments found. Creating new log.
I20260812 19:08:20.005839 21089 log.cc:824] T 00000000000000000000000000000000 P 041f6f0f9dc24fe28a299fdb2cc18ef7: Log is configured to *not* fsync() on all Append() calls
I20260812 19:08:20.008101 21089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 041f6f0f9dc24fe28a299fdb2cc18ef7: No bootstrap required, opened a new log
I20260812 19:08:20.009089 26120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:08:20.010627 20884 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "7fb91cae50d4433f854e33fc362c3ecb"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41846
I20260812 19:08:20.012039 21089 raft_consensus.cc:348] T 00000000000000000000000000000000 P 041f6f0f9dc24fe28a299fdb2cc18ef7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb91cae50d4433f854e33fc362c3ecb" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 35673 } } peers { permanent_uuid: "1f526cbb4af544f6b30cda3532ee16d1" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 35211 } } peers { permanent_uuid: "041f6f0f9dc24fe28a299fdb2cc18ef7" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 } }
I20260812 19:08:20.012312 21089 raft_consensus.cc:729] T 00000000000000000000000000000000 P 041f6f0f9dc24fe28a299fdb2cc18ef7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 041f6f0f9dc24fe28a299fdb2cc18ef7, State: Initialized, Role: FOLLOWER
I20260812 19:08:20.012688 21089 consensus_queue.cc:260] T 00000000000000000000000000000000 P 041f6f0f9dc24fe28a299fdb2cc18ef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb91cae50d4433f854e33fc362c3ecb" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 35673 } } peers { permanent_uuid: "1f526cbb4af544f6b30cda3532ee16d1" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 35211 } } peers { permanent_uuid: "041f6f0f9dc24fe28a299fdb2cc18ef7" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 } }
I20260812 19:08:20.013342 21120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 041f6f0f9dc24fe28a299fdb2cc18ef7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb91cae50d4433f854e33fc362c3ecb" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 35673 } } peers { permanent_uuid: "1f526cbb4af544f6b30cda3532ee16d1" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 35211 } } peers { permanent_uuid: "041f6f0f9dc24fe28a299fdb2cc18ef7" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 } } }
I20260812 19:08:20.013554 21120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 041f6f0f9dc24fe28a299fdb2cc18ef7 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:08:20.014218 21089 sys_catalog.cc:564] T 00000000000000000000000000000000 P 041f6f0f9dc24fe28a299fdb2cc18ef7 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 19:08:20.019881 21134 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 041f6f0f9dc24fe28a299fdb2cc18ef7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:08:20.019979 21134 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:08:20.017921 21005 master_runner.cc:186] Error getting master registration for 127.25.130.62:35673: OK, instance_id { permanent_uuid: "7fb91cae50d4433f854e33fc362c3ecb" instance_seqno: 1786561699477774 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 19:08:20.021889 21005 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 19:08:20.114282 20912 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 } attempt: 4
Aug 12 19:08:20 dist-test-slave-49h3 krb5kdc[20808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561699, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for kudu/127.25.130.60@KRBTEST.COM
I20260812 19:08:20.124225 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:20.114960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36038 (local address 127.25.130.60:40519)
0812 19:08:20.115416 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:20.115421 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:20.115436 (+ 15us) server_negotiation.cc:407] Connection header received
0812 19:08:20.115484 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:20.115487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:20.115532 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:20.115604 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:20.116085 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:20.116881 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:20.118046 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:20.118404 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:20.118496 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:20.120852 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:20.120856 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:20.120859 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:20.120888 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:20.122557 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:20.122970 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:20.122974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:20.122976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:20.123066 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:20.123356 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:20.123360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:20.123362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:20.123592 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:20.123690 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:20.123956 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 19:08:20.124076 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367}
I20260812 19:08:20.124225 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:20.114877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:40519 (local address 127.0.0.1:36038)
0812 19:08:20.115108 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:20.115124 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:08:20.115239 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:08:20.115654 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:08:20.115657 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:08:20.115853 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:08:20.115950 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:20.115956 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:20.117021 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:20.117025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:20.117907 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:20.117919 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:20.118515 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:20.118519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:20.118600 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:20.119150 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:08:20.119168 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:08:20.120686 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:08:20.122660 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:20.122664 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:20.122666 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:20.122875 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:20.123172 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:20.123175 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:20.123177 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:20.123269 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:20.123691 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:08:20.123694 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:08:20.123796 (+ 102us) client_negotiation.cc:769] Sending connection context
0812 19:08:20.123884 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 19:08:20.124035 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
I20260812 19:08:20.128123 20912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fb91cae50d4433f854e33fc362c3ecb: Bootstrap starting.
I20260812 19:08:20.131114 20912 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7fb91cae50d4433f854e33fc362c3ecb: Neither blocks nor log segments found. Creating new log.
I20260812 19:08:20.132069 20912 log.cc:824] T 00000000000000000000000000000000 P 7fb91cae50d4433f854e33fc362c3ecb: Log is configured to *not* fsync() on all Append() calls
I20260812 19:08:20.134502 20912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fb91cae50d4433f854e33fc362c3ecb: No bootstrap required, opened a new log
I20260812 19:08:20.137744 20912 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7fb91cae50d4433f854e33fc362c3ecb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb91cae50d4433f854e33fc362c3ecb" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 35673 } } peers { permanent_uuid: "1f526cbb4af544f6b30cda3532ee16d1" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 35211 } } peers { permanent_uuid: "041f6f0f9dc24fe28a299fdb2cc18ef7" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 } }
I20260812 19:08:20.138018 20912 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7fb91cae50d4433f854e33fc362c3ecb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fb91cae50d4433f854e33fc362c3ecb, State: Initialized, Role: FOLLOWER
I20260812 19:08:20.138367 20912 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7fb91cae50d4433f854e33fc362c3ecb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb91cae50d4433f854e33fc362c3ecb" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 35673 } } peers { permanent_uuid: "1f526cbb4af544f6b30cda3532ee16d1" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 35211 } } peers { permanent_uuid: "041f6f0f9dc24fe28a299fdb2cc18ef7" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 } }
I20260812 19:08:20.138945 21137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fb91cae50d4433f854e33fc362c3ecb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb91cae50d4433f854e33fc362c3ecb" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 35673 } } peers { permanent_uuid: "1f526cbb4af544f6b30cda3532ee16d1" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 35211 } } peers { permanent_uuid: "041f6f0f9dc24fe28a299fdb2cc18ef7" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 } } }
I20260812 19:08:20.139227 21137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fb91cae50d4433f854e33fc362c3ecb [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:08:20.139856 20912 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7fb91cae50d4433f854e33fc362c3ecb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:08:20.139892 20884 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7fb91cae50d4433f854e33fc362c3ecb [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:08:20.139988 20884 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7fb91cae50d4433f854e33fc362c3ecb [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:08:20.141398 20884 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7fb91cae50d4433f854e33fc362c3ecb [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb91cae50d4433f854e33fc362c3ecb" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 35673 } } peers { permanent_uuid: "1f526cbb4af544f6b30cda3532ee16d1" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 35211 } } peers { permanent_uuid: "041f6f0f9dc24fe28a299fdb2cc18ef7" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 } }
I20260812 19:08:20.143509 20968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7fb91cae50d4433f854e33fc362c3ecb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1f526cbb4af544f6b30cda3532ee16d1"
I20260812 19:08:20.143800 20968 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f526cbb4af544f6b30cda3532ee16d1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:08:20.142521 20884 leader_election.cc:288] T 00000000000000000000000000000000 P 7fb91cae50d4433f854e33fc362c3ecb [CANDIDATE]: Term 1 election: Requested vote from peers 1f526cbb4af544f6b30cda3532ee16d1 (127.25.130.61:35211), 041f6f0f9dc24fe28a299fdb2cc18ef7 (127.25.130.60:40519)
I20260812 19:08:20.145310 20968 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1f526cbb4af544f6b30cda3532ee16d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fb91cae50d4433f854e33fc362c3ecb in term 1.
I20260812 19:08:20.145890 21057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7fb91cae50d4433f854e33fc362c3ecb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "041f6f0f9dc24fe28a299fdb2cc18ef7"
I20260812 19:08:20.146189 21057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 041f6f0f9dc24fe28a299fdb2cc18ef7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:08:20.147480 21057 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 041f6f0f9dc24fe28a299fdb2cc18ef7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fb91cae50d4433f854e33fc362c3ecb in term 1.
I20260812 19:08:20.147977 20848 leader_election.cc:302] T 00000000000000000000000000000000 P 7fb91cae50d4433f854e33fc362c3ecb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 041f6f0f9dc24fe28a299fdb2cc18ef7, 7fb91cae50d4433f854e33fc362c3ecb; no voters:
I20260812 19:08:20.148248 21137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7fb91cae50d4433f854e33fc362c3ecb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:08:20.148466 21137 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7fb91cae50d4433f854e33fc362c3ecb [term 1 LEADER]: Becoming Leader. State: Replica: 7fb91cae50d4433f854e33fc362c3ecb, State: Running, Role: LEADER
I20260812 19:08:20.148816 21137 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7fb91cae50d4433f854e33fc362c3ecb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb91cae50d4433f854e33fc362c3ecb" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 35673 } } peers { permanent_uuid: "1f526cbb4af544f6b30cda3532ee16d1" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 35211 } } peers { permanent_uuid: "041f6f0f9dc24fe28a299fdb2cc18ef7" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 } }
W20260812 19:08:20.149312 21150 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7fb91cae50d4433f854e33fc362c3ecb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:08:20.149387 21150 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:08:20.150787 21137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fb91cae50d4433f854e33fc362c3ecb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7fb91cae50d4433f854e33fc362c3ecb. Latest consensus state: current_term: 1 leader_uuid: "7fb91cae50d4433f854e33fc362c3ecb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb91cae50d4433f854e33fc362c3ecb" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 35673 } } peers { permanent_uuid: "1f526cbb4af544f6b30cda3532ee16d1" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 35211 } } peers { permanent_uuid: "041f6f0f9dc24fe28a299fdb2cc18ef7" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 } } }
I20260812 19:08:20.150902 21137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fb91cae50d4433f854e33fc362c3ecb [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:20.153522 20825 master_runner.cc:263] No leader master found from master 7fb91cae50d4433f854e33fc362c3ecb
I20260812 19:08:20.153609 20825 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 19:08:20.156088 21160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:08:20.158953 21160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:08:20.169337 20915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:20.154105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41848 (local address 127.25.130.62:35673)
0812 19:08:20.154241 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:20.154246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:20.156289 (+ 2043us) server_negotiation.cc:407] Connection header received
0812 19:08:20.156489 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:20.156493 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:20.156541 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:20.156593 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:20.157702 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:20.158376 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:20.159570 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:20.159885 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:20.159964 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:20.165361 (+ 5397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:20.165365 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:20.165368 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:20.165404 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:20.167292 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:20.167848 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:20.167853 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:20.167855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:20.167948 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:20.168268 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:20.168272 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:20.168274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:20.168510 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:20.168616 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:20.169005 (+ 389us) server_negotiation.cc:299] Negotiation successful
0812 19:08:20.169144 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 19:08:20.173954 21057 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 041f6f0f9dc24fe28a299fdb2cc18ef7 [term 1 FOLLOWER]: Refusing update from remote peer 7fb91cae50d4433f854e33fc362c3ecb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:08:20.174865 21157 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7fb91cae50d4433f854e33fc362c3ecb [LEADER]: Connected to new peer: Peer: permanent_uuid: "041f6f0f9dc24fe28a299fdb2cc18ef7" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:08:20.178503 21000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:20.154692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57484 (local address 127.25.130.61:35211)
0812 19:08:20.154819 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:20.154824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:20.159651 (+ 4827us) server_negotiation.cc:407] Connection header received
0812 19:08:20.159866 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:20.159870 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:20.159925 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:20.159980 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:20.169583 (+ 9603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:20.170364 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:20.171569 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:20.171903 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:20.171988 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:20.173702 (+ 1714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:20.173706 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:20.173708 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:20.173737 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:20.176326 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:20.176833 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:20.176837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:20.176839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:20.176928 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:20.177215 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:20.177218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:20.177220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:20.177489 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:20.177610 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:20.178213 (+ 603us) server_negotiation.cc:299] Negotiation successful
0812 19:08:20.178346 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 19:08:20.180508 21120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 041f6f0f9dc24fe28a299fdb2cc18ef7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7fb91cae50d4433f854e33fc362c3ecb. Latest consensus state: current_term: 1 leader_uuid: "7fb91cae50d4433f854e33fc362c3ecb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb91cae50d4433f854e33fc362c3ecb" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 35673 } } peers { permanent_uuid: "1f526cbb4af544f6b30cda3532ee16d1" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 35211 } } peers { permanent_uuid: "041f6f0f9dc24fe28a299fdb2cc18ef7" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 } } }
I20260812 19:08:20.180647 21120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 041f6f0f9dc24fe28a299fdb2cc18ef7 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:08:20.181964 20968 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1f526cbb4af544f6b30cda3532ee16d1 [term 1 FOLLOWER]: Refusing update from remote peer 7fb91cae50d4433f854e33fc362c3ecb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:08:20.182674 21157 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7fb91cae50d4433f854e33fc362c3ecb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f526cbb4af544f6b30cda3532ee16d1" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 35211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:08:20.186856 21098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f526cbb4af544f6b30cda3532ee16d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7fb91cae50d4433f854e33fc362c3ecb. Latest consensus state: current_term: 1 leader_uuid: "7fb91cae50d4433f854e33fc362c3ecb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb91cae50d4433f854e33fc362c3ecb" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 35673 } } peers { permanent_uuid: "1f526cbb4af544f6b30cda3532ee16d1" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 35211 } } peers { permanent_uuid: "041f6f0f9dc24fe28a299fdb2cc18ef7" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 } } }
I20260812 19:08:20.186997 21098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f526cbb4af544f6b30cda3532ee16d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:08:20.187655 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:20.155434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36052 (local address 127.25.130.60:40519)
0812 19:08:20.155549 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:20.155554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:20.160920 (+ 5366us) server_negotiation.cc:407] Connection header received
0812 19:08:20.162356 (+ 1436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:20.162360 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:20.162416 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:20.162473 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:20.163322 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:20.164118 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:20.169690 (+ 5572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:20.170011 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:20.170094 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:20.173253 (+ 3159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:20.173257 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:20.173259 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:20.173290 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:20.175509 (+ 2219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:20.180204 (+ 4695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:20.180208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:20.180210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:20.180318 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:20.180633 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:20.180649 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:20.180651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:20.180880 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:20.181020 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:20.187331 (+ 6311us) server_negotiation.cc:299] Negotiation successful
0812 19:08:20.187489 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 19:08:20.190587 21137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fb91cae50d4433f854e33fc362c3ecb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7fb91cae50d4433f854e33fc362c3ecb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb91cae50d4433f854e33fc362c3ecb" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 35673 } } peers { permanent_uuid: "1f526cbb4af544f6b30cda3532ee16d1" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 35211 } } peers { permanent_uuid: "041f6f0f9dc24fe28a299fdb2cc18ef7" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 } } }
I20260812 19:08:20.190718 21137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fb91cae50d4433f854e33fc362c3ecb [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:20.191119 21167 mvcc.cc:205] Tried to move back new op lower bound from 7317756723903135744 to 7317756723813380096. Current Snapshot: MvccSnapshot[applied={T|T < 7317756723903135744}]
I20260812 19:08:20.191349 21120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 041f6f0f9dc24fe28a299fdb2cc18ef7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7fb91cae50d4433f854e33fc362c3ecb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb91cae50d4433f854e33fc362c3ecb" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 35673 } } peers { permanent_uuid: "1f526cbb4af544f6b30cda3532ee16d1" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 35211 } } peers { permanent_uuid: "041f6f0f9dc24fe28a299fdb2cc18ef7" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 } } }
I20260812 19:08:20.191462 21120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 041f6f0f9dc24fe28a299fdb2cc18ef7 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:08:20.192418 21169 mvcc.cc:205] Tried to move back new op lower bound from 7317756723903135744 to 7317756723813380096. Current Snapshot: MvccSnapshot[applied={T|T < 7317756723903135744}]
I20260812 19:08:20.194257 21157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fb91cae50d4433f854e33fc362c3ecb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7fb91cae50d4433f854e33fc362c3ecb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb91cae50d4433f854e33fc362c3ecb" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 35673 } } peers { permanent_uuid: "1f526cbb4af544f6b30cda3532ee16d1" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 35211 } } peers { permanent_uuid: "041f6f0f9dc24fe28a299fdb2cc18ef7" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 } } }
I20260812 19:08:20.194396 21157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fb91cae50d4433f854e33fc362c3ecb [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:20.198161 21098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f526cbb4af544f6b30cda3532ee16d1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7fb91cae50d4433f854e33fc362c3ecb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb91cae50d4433f854e33fc362c3ecb" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 35673 } } peers { permanent_uuid: "1f526cbb4af544f6b30cda3532ee16d1" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 35211 } } peers { permanent_uuid: "041f6f0f9dc24fe28a299fdb2cc18ef7" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 40519 } } }
I20260812 19:08:20.198282 21098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f526cbb4af544f6b30cda3532ee16d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:08:20.199229 21160 catalog_manager.cc:1347] Generated new cluster ID: 1ba1a820566e4b68a1e5089588d43c6f
I20260812 19:08:20.199301 21160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:08:20.220955 21160 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:08:20.222339 21160 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:08:20.232131 21160 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7fb91cae50d4433f854e33fc362c3ecb: Generated new TSK 0
I20260812 19:08:20.233183 21160 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 19:08:20.260454 20915 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:08:20.258903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41854 (local address 127.25.130.62:35673)
0812 19:08:20.259054 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:20.259059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:20.259246 (+ 187us) server_negotiation.cc:407] Connection header received
0812 19:08:20.259428 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:20.259433 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:20.259488 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:20.259563 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:20.260288 (+ 725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41854: BlockingRecv error: recv got EOF from 127.0.0.1:41854 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260812 19:08:20.261072 21000 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:08:20.259624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57494 (local address 127.25.130.61:35211)
0812 19:08:20.259805 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:20.259809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:20.260041 (+ 232us) server_negotiation.cc:407] Connection header received
0812 19:08:20.260227 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:20.260230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:20.260283 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:20.260333 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:20.260932 (+ 599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57494: BlockingRecv error: recv got EOF from 127.0.0.1:57494 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260812 19:08:20.261842 21091 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:08:20.260080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36066 (local address 127.25.130.60:40519)
0812 19:08:20.260214 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:20.260218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:20.260939 (+ 721us) server_negotiation.cc:407] Connection header received
0812 19:08:20.261026 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:20.261029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:20.261086 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:20.261177 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:20.261740 (+ 563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36066: BlockingRecv error: recv got EOF from 127.0.0.1:36066 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260812 19:08:20.264756 21000 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:08:20.263679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57500 (local address 127.25.130.61:35211)
0812 19:08:20.263804 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:20.263808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:20.263932 (+ 124us) server_negotiation.cc:407] Connection header received
0812 19:08:20.264091 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:20.264093 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:20.264161 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:20.264229 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:20.264635 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57500: BlockingRecv error: recv got EOF from 127.0.0.1:57500 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260812 19:08:20.269106 21000 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:08:20.268250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57504 (local address 127.25.130.61:35211)
0812 19:08:20.268381 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:20.268386 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:20.268399 (+ 13us) server_negotiation.cc:407] Connection header received
0812 19:08:20.268456 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:20.268459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:20.268510 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:20.268593 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:20.268989 (+ 396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57504: BlockingRecv error: recv got EOF from 127.0.0.1:57504 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260812 19:08:20.273743 21000 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:08:20.272714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57506 (local address 127.25.130.61:35211)
0812 19:08:20.272832 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:20.272836 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:20.272948 (+ 112us) server_negotiation.cc:407] Connection header received
0812 19:08:20.273072 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:20.273074 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:20.273117 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:20.273183 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:20.273629 (+ 446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57506: BlockingRecv error: recv got EOF from 127.0.0.1:57506 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260812 19:08:20.280949 21000 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:08:20.279845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57520 (local address 127.25.130.61:35211)
0812 19:08:20.280009 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:20.280013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:20.280125 (+ 112us) server_negotiation.cc:407] Connection header received
0812 19:08:20.280275 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:20.280277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:20.280326 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:20.280392 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:20.280823 (+ 431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57520: BlockingRecv error: recv got EOF from 127.0.0.1:57520 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260812 19:08:20.292505 21000 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:08:20.291323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57536 (local address 127.25.130.61:35211)
0812 19:08:20.291479 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:20.291485 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:20.291634 (+ 149us) server_negotiation.cc:407] Connection header received
0812 19:08:20.291753 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:20.291756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:20.291819 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:20.291925 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:20.292381 (+ 456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57536: BlockingRecv error: recv got EOF from 127.0.0.1:57536 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260812 19:08:20.311924 21000 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:08:20.310695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57546 (local address 127.25.130.61:35211)
0812 19:08:20.310873 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:20.310879 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:20.310992 (+ 113us) server_negotiation.cc:407] Connection header received
0812 19:08:20.311133 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:20.311146 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:20.311213 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:20.311320 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:20.311762 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57546: BlockingRecv error: recv got EOF from 127.0.0.1:57546 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260812 19:08:20.347146 21000 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:08:20.345948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57548 (local address 127.25.130.61:35211)
0812 19:08:20.346110 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:20.346115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:20.346254 (+ 139us) server_negotiation.cc:407] Connection header received
0812 19:08:20.346404 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:20.346406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:20.346458 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:20.346523 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:20.347004 (+ 481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57548: BlockingRecv error: recv got EOF from 127.0.0.1:57548 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260812 19:08:20.414585 21000 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:08:20.413371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57562 (local address 127.25.130.61:35211)
0812 19:08:20.413569 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:20.413574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:20.413715 (+ 141us) server_negotiation.cc:407] Connection header received
0812 19:08:20.413892 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:20.413895 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:20.413951 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:20.414012 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:20.414453 (+ 441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57562: BlockingRecv error: recv got EOF from 127.0.0.1:57562 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260812 19:08:20.546038 21000 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:08:20.544828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57564 (local address 127.25.130.61:35211)
0812 19:08:20.544991 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:20.544996 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:20.545096 (+ 100us) server_negotiation.cc:407] Connection header received
0812 19:08:20.545253 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:20.545256 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:20.545310 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:20.545382 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:20.545896 (+ 514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57564: BlockingRecv error: recv got EOF from 127.0.0.1:57564 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260812 19:08:20.805230 21000 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:08:20.804149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57572 (local address 127.25.130.61:35211)
0812 19:08:20.804303 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:20.804308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:20.804324 (+ 16us) server_negotiation.cc:407] Connection header received
0812 19:08:20.804490 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:20.804494 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:20.804550 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:20.804623 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:20.805100 (+ 477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57572: BlockingRecv error: recv got EOF from 127.0.0.1:57572 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 19:08:21.002125 21114 catalog_manager.cc:1259] Loaded cluster ID: 1ba1a820566e4b68a1e5089588d43c6f
I20260812 19:08:21.002224 21114 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1f526cbb4af544f6b30cda3532ee16d1: loading cluster ID for follower catalog manager: success
I20260812 19:08:21.003896 21114 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1f526cbb4af544f6b30cda3532ee16d1: acquiring CA information for follower catalog manager: success
I20260812 19:08:21.005075 21114 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1f526cbb4af544f6b30cda3532ee16d1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 19:08:21.021312 21134 catalog_manager.cc:1259] Loaded cluster ID: 1ba1a820566e4b68a1e5089588d43c6f
I20260812 19:08:21.021405 21134 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 041f6f0f9dc24fe28a299fdb2cc18ef7: loading cluster ID for follower catalog manager: success
I20260812 19:08:21.023161 21134 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 041f6f0f9dc24fe28a299fdb2cc18ef7: acquiring CA information for follower catalog manager: success
I20260812 19:08:21.024358 21134 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 041f6f0f9dc24fe28a299fdb2cc18ef7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 19:08:21.323129 21263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:21.319281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57582 (local address 127.25.130.61:35211)
0812 19:08:21.319564 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:21.319568 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:21.319586 (+ 18us) server_negotiation.cc:407] Connection header received
0812 19:08:21.319670 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:21.319674 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:21.319816 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:21.319916 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:08:21.320194 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:21.320729 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:21.321640 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:21.321874 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:21.321962 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:21.322277 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:08:21.322726 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:08:21.322838 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:21.322908 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 19:08:21.322971 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":91,"threads_started":1}
I20260812 19:08:21.329386 21269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:21.326161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36068 (local address 127.25.130.60:40519)
0812 19:08:21.326403 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:21.326407 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:21.326421 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:08:21.326504 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:21.326507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:21.326631 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:21.326729 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:08:21.326885 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:21.327401 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:21.328142 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:21.328359 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:21.328431 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:21.328652 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:08:21.329081 (+ 429us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:08:21.329176 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:21.329201 (+ 25us) server_negotiation.cc:299] Negotiation successful
0812 19:08:21.329256 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":71,"threads_started":1}
I20260812 19:08:21.336469 21269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:21.332819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36070 (local address 127.25.130.60:40519)
0812 19:08:21.332927 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:21.332931 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:21.333091 (+ 160us) server_negotiation.cc:407] Connection header received
0812 19:08:21.333279 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:21.333282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:21.333335 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:21.333401 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:08:21.333644 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:21.334129 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:21.335300 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:21.335616 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:21.335703 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:21.336026 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:08:21.336109 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:08:21.336152 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:21.336316 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 19:08:21.336367 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 19:08:21.336452 21263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:21.332382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57588 (local address 127.25.130.61:35211)
0812 19:08:21.332515 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:21.332519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:21.332786 (+ 267us) server_negotiation.cc:407] Connection header received
0812 19:08:21.332981 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:21.332984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:21.333031 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:21.333091 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:08:21.333343 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:21.334147 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:21.335249 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:21.335599 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:21.335703 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:21.335978 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:08:21.336067 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:08:21.336137 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:21.336246 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 19:08:21.336311 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 19:08:21.337183 21278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:21.332498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41856 (local address 127.25.130.62:35673)
0812 19:08:21.333059 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:21.333063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:21.333077 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:08:21.333180 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:21.333183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:21.333358 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:21.333776 (+ 418us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:08:21.333889 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:21.334627 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:21.335581 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:21.335910 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:21.336002 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:21.336303 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:08:21.336630 (+ 327us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:08:21.336713 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:21.336984 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 19:08:21.337049 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":83,"threads_started":1}
I20260812 19:08:21.340068 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 20825
I20260812 19:08:21.349685 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 20919
I20260812 19:08:21.357191 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 21005
2026-08-12T19:08:21Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3297 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:08:21 dist-test-slave-49h3 krb5kdc[21284](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:08:21 dist-test-slave-49h3 krb5kdc[21284](info): set up 2 sockets
Aug 12 19:08:21 dist-test-slave-49h3 krb5kdc[21284](info): commencing operation
krb5kdc: starting...
W20260812 19:08:22.414517 26120 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/krb5kdc/test-user.keytab.
Aug 12 19:08:22 dist-test-slave-49h3 krb5kdc[21284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:08:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:08:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
Entry for principal kudu/127.25.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab.
I20260812 19:08:22.541747 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:42001
--webserver_interface=127.25.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:38103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:42001,127.25.130.61:43423,127.25.130.60:34691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:08:22.655616 21300 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 19:08:22.655947 21300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:08:22.655993 21300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:08:22.656057 21300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:08:22.660228 21300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:08:22.660306 21300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:08:22.660336 21300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:08:22.660357 21300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:08:22.660393 21300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:08:22.666049 21300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:38103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:42001,127.25.130.61:43423,127.25.130.60:34691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.62.keytab
--principal=kudu/127.25.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.62:42001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:08:22.667194 21300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:08:22.668252 21300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:08:22.677762 21311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:08:22.677800 21307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:08:22.677800 21308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:22.678081 21300 server_base.cc:1034] running on GCE node
I20260812 19:08:22.678678 21300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:08:22.679811 21300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:08:22.680976 21300 hybrid_clock.cc:648] HybridClock initialized: now 1786561702680959 us; error 36 us; skew 500 ppm
Aug 12 19:08:22 dist-test-slave-49h3 krb5kdc[21284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561702, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:08:22.687700 21300 init.cc:377] Logged in from keytab as kudu/127.25.130.62@KRBTEST.COM (short username kudu)
I20260812 19:08:22.688694 21300 webserver.cc:460] Webserver started at http://127.25.130.62:38055/ using document root <none> and password file <none>
I20260812 19:08:22.689229 21300 fs_manager.cc:359] Metadata directory not provided
I20260812 19:08:22.689289 21300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:08:22.689517 21300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:08:22.691615 21300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-0/data/instance:
uuid: "6f34b8244a20472ea91e8b155f7b65e5"
format_stamp: "Formatted at 2026-08-12 19:08:22 on dist-test-slave-49h3"
server_key: "c7209171c1887ddfa28bafa9390174ee"
server_key_iv: "924a9c613ee0873642fbb8258bdfdccc"
server_key_version: "encryptionkey@0"
I20260812 19:08:22.692157 21300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-0/wal/instance:
uuid: "6f34b8244a20472ea91e8b155f7b65e5"
format_stamp: "Formatted at 2026-08-12 19:08:22 on dist-test-slave-49h3"
server_key: "c7209171c1887ddfa28bafa9390174ee"
server_key_iv: "924a9c613ee0873642fbb8258bdfdccc"
server_key_version: "encryptionkey@0"
I20260812 19:08:22.695958 21300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260812 19:08:22.698357 21318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:08:22.699572 21300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:08:22.699679 21300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-0/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-0/wal
uuid: "6f34b8244a20472ea91e8b155f7b65e5"
format_stamp: "Formatted at 2026-08-12 19:08:22 on dist-test-slave-49h3"
server_key: "c7209171c1887ddfa28bafa9390174ee"
server_key_iv: "924a9c613ee0873642fbb8258bdfdccc"
server_key_version: "encryptionkey@0"
I20260812 19:08:22.699769 21300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:08:22.709630 21300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:08:22.712597 21300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:08:22.712749 21300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:08:22.719676 21300 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.62:42001
I20260812 19:08:22.719936 21381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.62:42001 every 8 connection(s)
I20260812 19:08:22.720640 21300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-0/data/info.pb
I20260812 19:08:22.723400 21384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:08:22.727236 21384 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42001 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:08:22.729211 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 21300
I20260812 19:08:22.729336 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-0/wal/instance
I20260812 19:08:22.729631 26120 external_mini_cluster.cc:1428] Setting key ed0abb5beba257f588a18583132b5ec4
Aug 12 19:08:22 dist-test-slave-49h3 krb5kdc[21284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561702, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.130.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.61@KRBTEST.COM" created.
I20260812 19:08:22.743006 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:22.729627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:42001 (local address 127.0.0.1:35700)
0812 19:08:22.730216 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:22.730276 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 19:08:22.731304 (+ 1028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:08:22.732104 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:08:22.732108 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:08:22.732432 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:08:22.732695 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:22.732708 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:22.734271 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:22.734274 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:22.735079 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:22.735089 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:22.735972 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:22.735974 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:22.736050 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:22.736712 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:08:22.736729 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:08:22.738395 (+ 1666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:08:22.740223 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:22.740232 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:22.740241 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:22.740463 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:22.740729 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:22.740731 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:22.740732 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:22.740833 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:22.741470 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:08:22.741478 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:08:22.741619 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 19:08:22.741767 (+ 148us) client_negotiation.cc:240] Negotiation successful
0812 19:08:22.742120 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":3,"spinlock_wait_cycles":7552,"thread_start_us":136,"threads_started":1}
I20260812 19:08:22.743016 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:22.730409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35700 (local address 127.25.130.62:42001)
0812 19:08:22.730665 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:22.730675 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:22.730711 (+ 36us) server_negotiation.cc:407] Connection header received
0812 19:08:22.731620 (+ 909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:22.731626 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:22.731889 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:22.732077 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:22.732846 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:22.734161 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:22.735206 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:22.735832 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:22.736005 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:22.738569 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:22.738578 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:22.738592 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:22.738618 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:22.740118 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:22.740552 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:22.740556 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:22.740558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:22.740663 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:22.740896 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:22.740899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:22.740900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:22.741354 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:22.741490 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:22.741877 (+ 387us) server_negotiation.cc:299] Negotiation successful
0812 19:08:22.742179 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":123,"server-negotiator.queue_time_us":172,"thread_start_us":68,"threads_started":1}
I20260812 19:08:22.745579 21384 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.61" port: 43423 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:08:22.746335 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:22.746075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:43423 (local address 127.0.0.1:38102)
0812 19:08:22.746197 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:22.746271 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.61:43423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 19:08:22.746554 21322 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.130.61:43423: connect: Connection refused (error 111)
I20260812 19:08:22.747375 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:22.747188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:43423 (local address 127.0.0.1:38104)
0812 19:08:22.747277 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:22.747321 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.61:43423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 19:08:22.747511 21384 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.130.61:43423: Network error: Client connection negotiation failed: client connection to 127.25.130.61:43423: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.61.keytab.
Entry for principal kudu/127.25.130.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.61.keytab.
I20260812 19:08:22.757827 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.61:43423
--webserver_interface=127.25.130.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.61.keytab
--principal=kudu/127.25.130.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:38103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:42001,127.25.130.61:43423,127.25.130.60:34691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:08:22.796769 21384 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.130.61" port: 43423 } attempt: 1
I20260812 19:08:22.797662 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:22.797327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:43423 (local address 127.0.0.1:38108)
0812 19:08:22.797504 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:22.797578 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.61:43423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 19:08:22.798413 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:22.798218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:43423 (local address 127.0.0.1:38116)
0812 19:08:22.798310 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:22.798356 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.61:43423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 19:08:22.798542 21384 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.130.61:43423: Network error: Client connection negotiation failed: client connection to 127.25.130.61:43423: connect: Connection refused (error 111)
I20260812 19:08:22.866699 21384 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.130.61" port: 43423 } attempt: 2
I20260812 19:08:22.867540 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:22.867240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:43423 (local address 127.0.0.1:38126)
0812 19:08:22.867386 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:22.867450 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.61:43423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 19:08:22.868290 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:22.868095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:43423 (local address 127.0.0.1:38138)
0812 19:08:22.868186 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:22.868233 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.61:43423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 19:08:22.868418 21384 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.130.61:43423: Network error: Client connection negotiation failed: client connection to 127.25.130.61:43423: connect: Connection refused (error 111)
W20260812 19:08:22.873618 21391 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 19:08:22.873931 21391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:08:22.873974 21391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:08:22.874035 21391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:08:22.878221 21391 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:08:22.878294 21391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:08:22.878314 21391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:08:22.878334 21391 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:08:22.878363 21391 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:08:22.883944 21391 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:38103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:42001,127.25.130.61:43423,127.25.130.60:34691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.61.keytab
--principal=kudu/127.25.130.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.61:43423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.25.130.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:08:22.885058 21391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:08:22.886154 21391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:08:22.894706 21398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:08:22.894706 21399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:08:22.894709 21401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:22.894788 21391 server_base.cc:1034] running on GCE node
I20260812 19:08:22.895571 21391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:08:22.896780 21391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:08:22.897943 21391 hybrid_clock.cc:648] HybridClock initialized: now 1786561702897926 us; error 33 us; skew 500 ppm
Aug 12 19:08:22 dist-test-slave-49h3 krb5kdc[21284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561702, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:08:22.900770 21391 init.cc:377] Logged in from keytab as kudu/127.25.130.61@KRBTEST.COM (short username kudu)
I20260812 19:08:22.901789 21391 webserver.cc:460] Webserver started at http://127.25.130.61:38235/ using document root <none> and password file <none>
I20260812 19:08:22.902390 21391 fs_manager.cc:359] Metadata directory not provided
I20260812 19:08:22.902463 21391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:08:22.902727 21391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:08:22.904979 21391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-1/data/instance:
uuid: "b03281a52cfa47dd99ccf9db9d55dbcb"
format_stamp: "Formatted at 2026-08-12 19:08:22 on dist-test-slave-49h3"
server_key: "757754fd52f834cadb2951ac2473a60a"
server_key_iv: "68d6b3b1341163a958dd4b8613048b35"
server_key_version: "encryptionkey@0"
I20260812 19:08:22.905587 21391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-1/wal/instance:
uuid: "b03281a52cfa47dd99ccf9db9d55dbcb"
format_stamp: "Formatted at 2026-08-12 19:08:22 on dist-test-slave-49h3"
server_key: "757754fd52f834cadb2951ac2473a60a"
server_key_iv: "68d6b3b1341163a958dd4b8613048b35"
server_key_version: "encryptionkey@0"
I20260812 19:08:22.909497 21391 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:08:22.911984 21410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:08:22.913210 21391 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:08:22.913333 21391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-1/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-1/wal
uuid: "b03281a52cfa47dd99ccf9db9d55dbcb"
format_stamp: "Formatted at 2026-08-12 19:08:22 on dist-test-slave-49h3"
server_key: "757754fd52f834cadb2951ac2473a60a"
server_key_iv: "68d6b3b1341163a958dd4b8613048b35"
server_key_version: "encryptionkey@0"
I20260812 19:08:22.913439 21391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:08:22.926186 21391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:08:22.929286 21391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:08:22.929528 21391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:08:22.936722 21391 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.61:43423
I20260812 19:08:22.936993 21475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.61:43423 every 8 connection(s)
I20260812 19:08:22.937768 21391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-1/data/info.pb
I20260812 19:08:22.940567 21476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:08:22.944519 21476 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42001 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:08:22.945116 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 21391
I20260812 19:08:22.945250 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-1/wal/instance
I20260812 19:08:22.945554 26120 external_mini_cluster.cc:1428] Setting key 5f5d7ed778d21ee0f1037b860e598c20
WARNING: no policy specified for kudu/127.25.130.60@KRBTEST.COM; defaulting to no policy
Aug 12 19:08:22 dist-test-slave-49h3 krb5kdc[21284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561702, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.61@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.130.60@KRBTEST.COM" created.
I20260812 19:08:22.959367 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:22.946624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35706 (local address 127.25.130.62:42001)
0812 19:08:22.946756 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:22.946760 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:22.947902 (+ 1142us) server_negotiation.cc:407] Connection header received
0812 19:08:22.949123 (+ 1221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:22.949127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:22.949192 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:22.949283 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:22.950495 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:22.951319 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:22.952472 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:22.952843 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:22.952954 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:22.955378 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:22.955381 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:22.955383 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:22.955412 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:22.956824 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:22.957660 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:22.957664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:22.957666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:22.957774 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:22.958138 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:22.958141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:22.958143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:22.958381 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:22.958475 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:22.959052 (+ 577us) server_negotiation.cc:299] Negotiation successful
0812 19:08:22.959193 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 19:08:22.959594 21384 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.130.61" port: 43423 } attempt: 3
I20260812 19:08:22.960739 21478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:22.947104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:42001 (local address 127.0.0.1:35706)
0812 19:08:22.947708 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:22.947771 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 19:08:22.948858 (+ 1087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:08:22.949433 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:08:22.949457 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:08:22.949843 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:08:22.950331 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:22.950345 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:22.951448 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:22.951451 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:22.952348 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:22.952358 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:22.952982 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:22.952985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:22.953139 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:22.953745 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:08:22.953762 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:08:22.955205 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:08:22.956963 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:22.956969 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:22.957110 (+ 141us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:22.957549 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:22.957883 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:22.957885 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:22.957887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:22.958043 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:22.958476 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:08:22.958482 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:08:22.958788 (+ 306us) client_negotiation.cc:769] Sending connection context
0812 19:08:22.958950 (+ 162us) client_negotiation.cc:240] Negotiation successful
0812 19:08:22.959256 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":156,"threads_started":1}
I20260812 19:08:22.962255 21476 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.61" port: 43423 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 19:08:22 dist-test-slave-49h3 krb5kdc[21284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561702, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for kudu/127.25.130.61@KRBTEST.COM
Aug 12 19:08:22 dist-test-slave-49h3 krb5kdc[21284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561702, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.61@KRBTEST.COM for kudu/127.25.130.61@KRBTEST.COM
I20260812 19:08:22.970357 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:22.960063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:43423 (local address 127.0.0.1:38148)
0812 19:08:22.960267 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:22.960281 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:08:22.960361 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:08:22.960985 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:08:22.960988 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:08:22.961162 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:08:22.961299 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:22.961306 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:22.962716 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:22.962720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:22.963893 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:22.963903 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:22.964539 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:22.964543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:22.964651 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:22.965157 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:08:22.965172 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:08:22.966869 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:08:22.968687 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:22.968690 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:22.968692 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:22.968890 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:22.969175 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:22.969178 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:22.969180 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:22.969262 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:22.969793 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:08:22.969796 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:08:22.969906 (+ 110us) client_negotiation.cc:769] Sending connection context
0812 19:08:22.969997 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 19:08:22.970165 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130}
I20260812 19:08:22.971033 21481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:22.960211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38148 (local address 127.25.130.61:43423)
0812 19:08:22.960606 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:22.960616 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:22.960634 (+ 18us) server_negotiation.cc:407] Connection header received
0812 19:08:22.960670 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:22.960674 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:22.960835 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:22.961468 (+ 633us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:22.961671 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:22.962571 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:22.964029 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:22.964431 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:22.964521 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:22.967008 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:22.967020 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:22.967039 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:22.967060 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:22.968562 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:22.968994 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:22.969000 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:22.969002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:22.969086 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:22.969344 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:22.969347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:22.969349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:22.969688 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:22.969816 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:22.970593 (+ 777us) server_negotiation.cc:299] Negotiation successful
0812 19:08:22.970753 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":150,"threads_started":1}
I20260812 19:08:22.972921 21478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:22.962783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:43423 (local address 127.0.0.1:38160)
0812 19:08:22.962903 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:22.962919 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:08:22.962994 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:08:22.963566 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:08:22.963570 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:08:22.963768 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:08:22.963895 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:22.963903 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:22.964957 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:22.964960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:22.965578 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:22.965587 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:22.966128 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:22.966132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:22.966234 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:22.966754 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:08:22.966769 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:08:22.969727 (+ 2958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:08:22.971482 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:22.971484 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:22.971486 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:22.971674 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:22.971940 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:22.971942 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:22.971944 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:22.972038 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:22.972380 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:08:22.972383 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:08:22.972479 (+ 96us) client_negotiation.cc:769] Sending connection context
0812 19:08:22.972598 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 19:08:22.972739 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"spinlock_wait_cycles":1408}
I20260812 19:08:22.973014 21482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:22.963005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38160 (local address 127.25.130.61:43423)
0812 19:08:22.963329 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:22.963348 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:22.963360 (+ 12us) server_negotiation.cc:407] Connection header received
0812 19:08:22.963389 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:22.963392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:22.963429 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:22.963515 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:22.964049 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:22.964848 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:22.965681 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:22.965986 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:22.966101 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:22.969896 (+ 3795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:22.969899 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:22.969901 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:22.969926 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:22.971389 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:22.971761 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:22.971764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:22.971766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:22.971850 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:22.972127 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:22.972129 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:22.972130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:22.972299 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:22.972381 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:22.972644 (+ 263us) server_negotiation.cc:299] Negotiation successful
0812 19:08:22.972801 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":266,"thread_start_us":162,"threads_started":1}
I20260812 19:08:22.973138 21384 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.60" port: 34691 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:08:22.973651 21476 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.60" port: 34691 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:08:22.973937 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:22.973669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:34691 (local address 127.0.0.1:53446)
0812 19:08:22.973806 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:22.973864 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:34691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 19:08:22.974411 21478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:22.974093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:34691 (local address 127.0.0.1:53448)
0812 19:08:22.974233 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:22.974342 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:34691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 19:08:22.974622 21415 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.130.60:34691: connect: Connection refused (error 111)
I20260812 19:08:22.975255 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:22.974409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:34691 (local address 127.0.0.1:53462)
0812 19:08:22.975156 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:22.975202 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:34691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":686}
W20260812 19:08:22.975378 21384 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.130.60:34691: Network error: Client connection negotiation failed: client connection to 127.25.130.60:34691: connect: Connection refused (error 111)
I20260812 19:08:22.975579 21478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:22.975302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:34691 (local address 127.0.0.1:53466)
0812 19:08:22.975433 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:22.975497 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:34691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260812 19:08:22.975701 21476 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.130.60:34691: Network error: Client connection negotiation failed: client connection to 127.25.130.60:34691: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.130.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.60.keytab.
Entry for principal kudu/127.25.130.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.60.keytab.
I20260812 19:08:22.976831 26120 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
/tmp/dist-test-taskt2JiBs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.60:34691
--webserver_interface=127.25.130.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.60.keytab
--principal=kudu/127.25.130.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.130.20:38103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.130.62:42001,127.25.130.61:43423,127.25.130.60:34691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:08:23.006538 21384 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.130.60" port: 34691 } attempt: 1
I20260812 19:08:23.007581 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:23.007192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:34691 (local address 127.0.0.1:53468)
0812 19:08:23.007383 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:23.007462 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:34691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260812 19:08:23.008334 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:23.008088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:34691 (local address 127.0.0.1:53470)
0812 19:08:23.008207 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:23.008264 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:34691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 19:08:23.008473 21384 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.130.60:34691: Network error: Client connection negotiation failed: client connection to 127.25.130.60:34691: connect: Connection refused (error 111)
I20260812 19:08:23.024959 21476 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.130.60" port: 34691 } attempt: 1
I20260812 19:08:23.025830 21478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:23.025544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:34691 (local address 127.0.0.1:53480)
0812 19:08:23.025684 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:23.025748 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:34691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 19:08:23.026604 21478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:23.026402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:34691 (local address 127.0.0.1:53496)
0812 19:08:23.026503 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:23.026547 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:34691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 19:08:23.026724 21476 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.130.60:34691: Network error: Client connection negotiation failed: client connection to 127.25.130.60:34691: connect: Connection refused (error 111)
I20260812 19:08:23.083664 21384 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.130.60" port: 34691 } attempt: 2
I20260812 19:08:23.084614 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:23.084246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:34691 (local address 127.0.0.1:53512)
0812 19:08:23.084423 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:23.084500 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:34691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260812 19:08:23.085343 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:23.085110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:34691 (local address 127.0.0.1:53514)
0812 19:08:23.085215 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:23.085269 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:34691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 19:08:23.085507 21384 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.130.60:34691: Network error: Client connection negotiation failed: client connection to 127.25.130.60:34691: connect: Connection refused (error 111)
W20260812 19:08:23.092903 21484 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 19:08:23.093212 21484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:08:23.093243 21484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:08:23.093283 21484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 19:08:23.094868 21476 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.130.60" port: 34691 } attempt: 2
I20260812 19:08:23.095727 21478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:23.095426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:34691 (local address 127.0.0.1:53528)
0812 19:08:23.095574 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:23.095639 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:34691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 19:08:23.096413 21478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:23.096189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:34691 (local address 127.0.0.1:53534)
0812 19:08:23.096294 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:23.096353 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.130.60:34691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 19:08:23.096537 21476 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.130.60:34691: Network error: Client connection negotiation failed: client connection to 127.25.130.60:34691: connect: Connection refused (error 111)
W20260812 19:08:23.097854 21484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:08:23.097913 21484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:08:23.097934 21484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:08:23.097965 21484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:08:23.097996 21484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:08:23.103700 21484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.130.20:38103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.25.130.62:42001,127.25.130.61:43423,127.25.130.60:34691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/krb5kdc/kudu_127.25.130.60.keytab
--principal=kudu/127.25.130.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.130.60:34691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.25.130.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:08:23.104880 21484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:08:23.105993 21484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:08:23.114827 21490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:23.114900 21484 server_base.cc:1034] running on GCE node
W20260812 19:08:23.114835 21491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:08:23.114835 21493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:08:23.115703 21484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:08:23.116958 21484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:08:23.118124 21484 hybrid_clock.cc:648] HybridClock initialized: now 1786561703118098 us; error 41 us; skew 500 ppm
Aug 12 19:08:23 dist-test-slave-49h3 krb5kdc[21284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561703, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:08:23.120921 21484 init.cc:377] Logged in from keytab as kudu/127.25.130.60@KRBTEST.COM (short username kudu)
I20260812 19:08:23.121949 21484 webserver.cc:460] Webserver started at http://127.25.130.60:44271/ using document root <none> and password file <none>
I20260812 19:08:23.122502 21484 fs_manager.cc:359] Metadata directory not provided
I20260812 19:08:23.122584 21484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:08:23.122799 21484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:08:23.124909 21484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-2/data/instance:
uuid: "f2f6c9b95e7a4ce998922cf4190f2b09"
format_stamp: "Formatted at 2026-08-12 19:08:23 on dist-test-slave-49h3"
server_key: "c917971a426c7a9eb983c591260f4d95"
server_key_iv: "8f209d3b880483210f1a59b2d14744bd"
server_key_version: "encryptionkey@0"
I20260812 19:08:23.125520 21484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-2/wal/instance:
uuid: "f2f6c9b95e7a4ce998922cf4190f2b09"
format_stamp: "Formatted at 2026-08-12 19:08:23 on dist-test-slave-49h3"
server_key: "c917971a426c7a9eb983c591260f4d95"
server_key_iv: "8f209d3b880483210f1a59b2d14744bd"
server_key_version: "encryptionkey@0"
I20260812 19:08:23.129258 21484 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:08:23.131682 21502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:08:23.132947 21484 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:08:23.133072 21484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-2/data,/tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-2/wal
uuid: "f2f6c9b95e7a4ce998922cf4190f2b09"
format_stamp: "Formatted at 2026-08-12 19:08:23 on dist-test-slave-49h3"
server_key: "c917971a426c7a9eb983c591260f4d95"
server_key_iv: "8f209d3b880483210f1a59b2d14744bd"
server_key_version: "encryptionkey@0"
I20260812 19:08:23.133179 21484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:08:23.155071 21484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:08:23.158195 21484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:08:23.158488 21484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:08:23.165936 21484 rpc_server.cc:307] RPC server started. Bound to: 127.25.130.60:34691
I20260812 19:08:23.166205 21566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.130.60:34691 every 8 connection(s)
I20260812 19:08:23.166960 21484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-2/data/info.pb
I20260812 19:08:23.169762 21567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:08:23.173609 21567 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42001 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:08:23.174019 26120 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu as pid 21484
I20260812 19:08:23.174168 26120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt2JiBs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561415988404-26120-0/minicluster-data/master-2/wal/instance
I20260812 19:08:23.174458 26120 external_mini_cluster.cc:1428] Setting key e33dbd30684650b493a9efbb0c2567bf
Aug 12 19:08:23 dist-test-slave-49h3 krb5kdc[21284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561703, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.60@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
Aug 12 19:08:23 dist-test-slave-49h3 krb5kdc[21284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.60@KRBTEST.COM
I20260812 19:08:23.184696 21384 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.130.60" port: 34691 } attempt: 3
I20260812 19:08:23.186947 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:23.175709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35720 (local address 127.25.130.62:42001)
0812 19:08:23.175860 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:23.175865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:23.176442 (+ 577us) server_negotiation.cc:407] Connection header received
0812 19:08:23.177247 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:23.177250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:23.177304 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:23.177379 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:23.178330 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.178832 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.179943 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.180265 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.180348 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:23.182985 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:23.182990 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:23.182993 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:23.183046 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:23.184540 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:23.185173 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:23.185176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:23.185178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:23.185271 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:23.185664 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:23.185667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:23.185669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:23.185954 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:23.186079 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:23.186649 (+ 570us) server_negotiation.cc:299] Negotiation successful
0812 19:08:23.186780 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 19:08:23.187692 21476 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.130.60" port: 34691 } attempt: 3
I20260812 19:08:23.188436 21572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:23.175811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.62:42001 (local address 127.0.0.1:35720)
0812 19:08:23.176312 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:23.176351 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 19:08:23.177051 (+ 700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:08:23.177481 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:08:23.177487 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:08:23.177920 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:08:23.178176 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:23.178200 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.178938 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.178941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:23.179816 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:23.179824 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.180423 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.180427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:23.180604 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:23.181205 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:08:23.181221 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:08:23.182788 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:08:23.184654 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:23.184665 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:23.184677 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:23.185061 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:23.185381 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:23.185384 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:23.185386 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:23.185581 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:23.186073 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:08:23.186080 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:08:23.186394 (+ 314us) client_negotiation.cc:769] Sending connection context
0812 19:08:23.187152 (+ 758us) client_negotiation.cc:240] Negotiation successful
0812 19:08:23.187424 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":23,"spinlock_wait_cycles":20864,"thread_start_us":164,"threads_started":1}
I20260812 19:08:23.189167 21570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:23.175810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53548 (local address 127.25.130.60:34691)
0812 19:08:23.176240 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:23.176258 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:23.176298 (+ 40us) server_negotiation.cc:407] Connection header received
0812 19:08:23.177043 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:23.177061 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:23.177260 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:23.177412 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:23.178266 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.179228 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.180155 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.180614 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.180717 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:23.183340 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:23.183346 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:23.183357 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:23.183378 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:23.184910 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:23.185395 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:23.185401 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:23.185403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:23.185620 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:23.185921 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:23.185925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:23.185927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:23.186426 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:23.186651 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:23.186876 (+ 225us) server_negotiation.cc:299] Negotiation successful
0812 19:08:23.187539 (+ 663us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":126,"server-negotiator.queue_time_us":295,"spinlock_wait_cycles":30464,"thread_start_us":137,"threads_started":1}
I20260812 19:08:23.189986 21567 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.61" port: 43423 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 19:08:23 dist-test-slave-49h3 krb5kdc[21284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561702, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.62@KRBTEST.COM for kudu/127.25.130.60@KRBTEST.COM
Aug 12 19:08:23 dist-test-slave-49h3 krb5kdc[21284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561702, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.61@KRBTEST.COM for kudu/127.25.130.60@KRBTEST.COM
Aug 12 19:08:23 dist-test-slave-49h3 krb5kdc[21284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561703, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.60@KRBTEST.COM for kudu/127.25.130.61@KRBTEST.COM
I20260812 19:08:23.196484 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:23.185176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:34691 (local address 127.0.0.1:53562)
0812 19:08:23.185380 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:23.185394 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:08:23.185496 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:08:23.185843 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:08:23.185846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:08:23.186001 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:08:23.186119 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:23.186126 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.187098 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.187102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:23.187857 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:23.187868 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.188474 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.188478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:23.188564 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:23.189060 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:08:23.189073 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:08:23.190609 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:08:23.193143 (+ 2534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:23.193147 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:23.193149 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:23.193406 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:23.193757 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:23.193760 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:23.193762 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:23.193851 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:23.195956 (+ 2105us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:08:23.195960 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:08:23.196080 (+ 120us) client_negotiation.cc:769] Sending connection context
0812 19:08:23.196168 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 19:08:23.196309 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125}
I20260812 19:08:23.197009 21575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:23.185245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53562 (local address 127.25.130.60:34691)
0812 19:08:23.185635 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:23.185644 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:23.185656 (+ 12us) server_negotiation.cc:407] Connection header received
0812 19:08:23.185686 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:23.185689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:23.185731 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:23.185803 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:23.186232 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.186951 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.188002 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.188341 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.188784 (+ 443us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:23.190750 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:23.190754 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:23.190756 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:23.190779 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:23.193005 (+ 2226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:23.193561 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:23.193564 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:23.193566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:23.193654 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:23.194979 (+ 1325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:23.194982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:23.194984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:23.195216 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:23.195312 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:23.196724 (+ 1412us) server_negotiation.cc:299] Negotiation successful
0812 19:08:23.196836 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":110,"threads_started":1}
Aug 12 19:08:23 dist-test-slave-49h3 krb5kdc[21284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.62@KRBTEST.COM
I20260812 19:08:23.198945 21478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:23.188130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:34691 (local address 127.0.0.1:53568)
0812 19:08:23.188392 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:23.188405 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:08:23.188487 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:08:23.189608 (+ 1121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:08:23.189612 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:08:23.189800 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:08:23.189918 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:23.189926 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.190954 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.190958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:23.191984 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:23.191997 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.192860 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.192864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:23.192960 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:23.193397 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:08:23.193411 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:08:23.194830 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:08:23.196857 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:23.196861 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:23.196863 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:23.197168 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:23.197880 (+ 712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:23.197883 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:23.197885 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:23.197991 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:23.198427 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:08:23.198430 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:08:23.198543 (+ 113us) client_negotiation.cc:769] Sending connection context
0812 19:08:23.198639 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 19:08:23.198797 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187}
I20260812 19:08:23.199620 21570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:23.188378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53568 (local address 127.25.130.60:34691)
0812 19:08:23.189333 (+ 955us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:23.189337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:23.189350 (+ 13us) server_negotiation.cc:407] Connection header received
0812 19:08:23.189382 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:23.189384 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:23.189423 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:23.190038 (+ 615us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:23.190129 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.190824 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.192134 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.192706 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.193975 (+ 1269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:23.194966 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:23.194970 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:23.194972 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:23.194993 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:23.196753 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:23.197670 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:23.197676 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:23.197678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:23.197771 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:23.198111 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:23.198114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:23.198115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:23.198318 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:23.199221 (+ 903us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:23.199293 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 19:08:23.199425 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":902,"spinlock_wait_cycles":1920,"thread_start_us":144,"threads_started":1}
I20260812 19:08:23.201020 21482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:23.190678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38170 (local address 127.25.130.61:43423)
0812 19:08:23.190792 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:23.190797 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:23.190939 (+ 142us) server_negotiation.cc:407] Connection header received
0812 19:08:23.191106 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:23.191109 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:23.191153 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:23.191199 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:23.191711 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.192350 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.193768 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.194126 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.194192 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:23.196909 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:23.196913 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:23.196916 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:23.196943 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:23.198927 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:23.199624 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:23.199627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:23.199629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:23.199720 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:23.200007 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:23.200010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:23.200012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:23.200313 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:23.200387 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:23.200753 (+ 366us) server_negotiation.cc:299] Negotiation successful
0812 19:08:23.200878 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 19:08:23.201592 21572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:23.190449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.61:43423 (local address 127.0.0.1:38170)
0812 19:08:23.190876 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:23.190889 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:08:23.190995 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:08:23.191323 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:08:23.191326 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:08:23.191488 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:08:23.191596 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:23.191604 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.192871 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.192875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:23.193646 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:23.193656 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.194335 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.194338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:23.194429 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:23.194815 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:08:23.194826 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:08:23.196096 (+ 1270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:08:23.199247 (+ 3151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:23.199250 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:23.199252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:23.199541 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:23.199799 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:23.199802 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:23.199804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:23.199893 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:23.200460 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:08:23.200463 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:08:23.200572 (+ 109us) client_negotiation.cc:769] Sending connection context
0812 19:08:23.201161 (+ 589us) client_negotiation.cc:240] Negotiation successful
0812 19:08:23.201305 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"spinlock_wait_cycles":7424}
I20260812 19:08:23.201812 21384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f34b8244a20472ea91e8b155f7b65e5: Bootstrap starting.
I20260812 19:08:23.202896 21476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b03281a52cfa47dd99ccf9db9d55dbcb: Bootstrap starting.
I20260812 19:08:23.203389 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:23.192505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35736 (local address 127.25.130.62:42001)
0812 19:08:23.192628 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:23.192632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:23.192644 (+ 12us) server_negotiation.cc:407] Connection header received
0812 19:08:23.192673 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:23.192676 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:23.192710 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:23.192770 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:23.193546 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.194238 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.195689 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.196006 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.196103 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:23.199145 (+ 3042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:23.199149 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:23.199152 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:23.199177 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:23.200934 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:23.201910 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:23.201914 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:23.201915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:23.202010 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:23.202346 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:23.202350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:23.202351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:23.202596 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:23.202817 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:23.203114 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 19:08:23.203244 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 19:08:23.203948 21567 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.130.60" port: 34691 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:08:23.205641 21384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f34b8244a20472ea91e8b155f7b65e5: Neither blocks nor log segments found. Creating new log.
I20260812 19:08:23.205948 21476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b03281a52cfa47dd99ccf9db9d55dbcb: Neither blocks nor log segments found. Creating new log.
I20260812 19:08:23.206614 21384 log.cc:824] T 00000000000000000000000000000000 P 6f34b8244a20472ea91e8b155f7b65e5: Log is configured to *not* fsync() on all Append() calls
I20260812 19:08:23.206806 21476 log.cc:824] T 00000000000000000000000000000000 P b03281a52cfa47dd99ccf9db9d55dbcb: Log is configured to *not* fsync() on all Append() calls
I20260812 19:08:23.208905 21384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f34b8244a20472ea91e8b155f7b65e5: No bootstrap required, opened a new log
Aug 12 19:08:23 dist-test-slave-49h3 krb5kdc[21284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561703, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.130.60@KRBTEST.COM for kudu/127.25.130.60@KRBTEST.COM
Aug 12 19:08:23 dist-test-slave-49h3 krb5kdc[21284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.130.61@KRBTEST.COM
I20260812 19:08:23.211360 21476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b03281a52cfa47dd99ccf9db9d55dbcb: No bootstrap required, opened a new log
I20260812 19:08:23.213135 21384 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f34b8244a20472ea91e8b155f7b65e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f34b8244a20472ea91e8b155f7b65e5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42001 } } peers { permanent_uuid: "b03281a52cfa47dd99ccf9db9d55dbcb" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 43423 } } peers { permanent_uuid: "f2f6c9b95e7a4ce998922cf4190f2b09" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 34691 } }
I20260812 19:08:23.213398 21384 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f34b8244a20472ea91e8b155f7b65e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f34b8244a20472ea91e8b155f7b65e5, State: Initialized, Role: FOLLOWER
I20260812 19:08:23.213624 21572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:23.204444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.130.60:34691 (local address 127.0.0.1:53584)
0812 19:08:23.205161 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0812 19:08:23.205181 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 19:08:23.205253 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:08:23.205566 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:08:23.205569 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:08:23.205742 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:08:23.205831 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:23.205838 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.206802 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.206804 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:23.207467 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:08:23.207477 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.207888 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.207892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:08:23.208004 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:23.208743 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:08:23.208754 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:08:23.210314 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:08:23.212152 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:23.212155 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:23.212157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:23.212382 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:23.212652 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:08:23.212654 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:08:23.212656 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:08:23.212728 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:08:23.213100 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:08:23.213103 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:08:23.213180 (+ 77us) client_negotiation.cc:769] Sending connection context
0812 19:08:23.213253 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 19:08:23.213391 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":15}
I20260812 19:08:23.213792 21384 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f34b8244a20472ea91e8b155f7b65e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f34b8244a20472ea91e8b155f7b65e5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42001 } } peers { permanent_uuid: "b03281a52cfa47dd99ccf9db9d55dbcb" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 43423 } } peers { permanent_uuid: "f2f6c9b95e7a4ce998922cf4190f2b09" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 34691 } }
I20260812 19:08:23.214356 21570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:23.204643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53584 (local address 127.25.130.60:34691)
0812 19:08:23.204763 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:23.204767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:23.205319 (+ 552us) server_negotiation.cc:407] Connection header received
0812 19:08:23.205344 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:23.205346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:23.205379 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:23.205434 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:23.205948 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.206691 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.207586 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.207797 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.207879 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:23.210466 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:23.210470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:23.210472 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:23.210493 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:23.212036 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:23.212486 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:23.212488 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:23.212490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:23.212577 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:23.212817 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:23.212819 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:23.212821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:23.213010 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:23.213909 (+ 899us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:23.213980 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 19:08:23.214205 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":45}
I20260812 19:08:23.215317 21384 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f34b8244a20472ea91e8b155f7b65e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:08:23.215394 21476 raft_consensus.cc:348] T 00000000000000000000000000000000 P b03281a52cfa47dd99ccf9db9d55dbcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f34b8244a20472ea91e8b155f7b65e5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42001 } } peers { permanent_uuid: "b03281a52cfa47dd99ccf9db9d55dbcb" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 43423 } } peers { permanent_uuid: "f2f6c9b95e7a4ce998922cf4190f2b09" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 34691 } }
I20260812 19:08:23.215592 21476 raft_consensus.cc:729] T 00000000000000000000000000000000 P b03281a52cfa47dd99ccf9db9d55dbcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b03281a52cfa47dd99ccf9db9d55dbcb, State: Initialized, Role: FOLLOWER
I20260812 19:08:23.215879 21476 consensus_queue.cc:260] T 00000000000000000000000000000000 P b03281a52cfa47dd99ccf9db9d55dbcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f34b8244a20472ea91e8b155f7b65e5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42001 } } peers { permanent_uuid: "b03281a52cfa47dd99ccf9db9d55dbcb" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 43423 } } peers { permanent_uuid: "f2f6c9b95e7a4ce998922cf4190f2b09" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 34691 } }
I20260812 19:08:23.216629 21482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:23.204977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38178 (local address 127.25.130.61:43423)
0812 19:08:23.205083 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:23.205087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:23.205101 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:08:23.205162 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:23.205165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:23.205208 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:23.205278 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:23.206256 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.206972 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.208075 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.208357 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.208453 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:23.211713 (+ 3260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:23.211717 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:23.211720 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:23.211746 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:23.213319 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:23.214264 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:23.214268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:23.214269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:23.214361 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:23.214855 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:23.214858 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:23.214860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:23.215071 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:23.215769 (+ 698us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:23.216343 (+ 574us) server_negotiation.cc:299] Negotiation successful
0812 19:08:23.216473 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 19:08:23.217239 21476 sys_catalog.cc:564] T 00000000000000000000000000000000 P b03281a52cfa47dd99ccf9db9d55dbcb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:08:23.217367 21567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2f6c9b95e7a4ce998922cf4190f2b09: Bootstrap starting.
I20260812 19:08:23.220249 21567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2f6c9b95e7a4ce998922cf4190f2b09: Neither blocks nor log segments found. Creating new log.
I20260812 19:08:23.221194 21567 log.cc:824] T 00000000000000000000000000000000 P f2f6c9b95e7a4ce998922cf4190f2b09: Log is configured to *not* fsync() on all Append() calls
I20260812 19:08:23.222523 26120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260812 19:08:23.223435 21607 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b03281a52cfa47dd99ccf9db9d55dbcb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:08:23.223572 21607 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:08:23.223785 21353 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "6f34b8244a20472ea91e8b155f7b65e5"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35736
I20260812 19:08:23.224025 21353 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f34b8244a20472ea91e8b155f7b65e5 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:08:23.224116 21353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f34b8244a20472ea91e8b155f7b65e5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:08:23.225595 21353 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f34b8244a20472ea91e8b155f7b65e5 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f34b8244a20472ea91e8b155f7b65e5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42001 } } peers { permanent_uuid: "b03281a52cfa47dd99ccf9db9d55dbcb" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 43423 } } peers { permanent_uuid: "f2f6c9b95e7a4ce998922cf4190f2b09" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 34691 } }
I20260812 19:08:23.226388 21353 leader_election.cc:288] T 00000000000000000000000000000000 P 6f34b8244a20472ea91e8b155f7b65e5 [CANDIDATE]: Term 1 election: Requested vote from peers b03281a52cfa47dd99ccf9db9d55dbcb (127.25.130.61:43423), f2f6c9b95e7a4ce998922cf4190f2b09 (127.25.130.60:34691)
I20260812 19:08:23.226825 21443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f34b8244a20472ea91e8b155f7b65e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b03281a52cfa47dd99ccf9db9d55dbcb"
I20260812 19:08:23.227068 21443 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b03281a52cfa47dd99ccf9db9d55dbcb [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:08:23.228478 21443 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b03281a52cfa47dd99ccf9db9d55dbcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f34b8244a20472ea91e8b155f7b65e5 in term 1.
I20260812 19:08:23.223404 21300 master_runner.cc:186] Error getting master registration for 127.25.130.60:34691: OK, instance_id { permanent_uuid: "f2f6c9b95e7a4ce998922cf4190f2b09" instance_seqno: 1786561703163959 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 19:08:23.228820 21300 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 19:08:23.226575 21391 master_runner.cc:186] Error getting master registration for 127.25.130.60:34691: OK, instance_id { permanent_uuid: "f2f6c9b95e7a4ce998922cf4190f2b09" instance_seqno: 1786561703163959 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 19:08:23.228922 21391 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 19:08:23.228932 21322 leader_election.cc:302] T 00000000000000000000000000000000 P 6f34b8244a20472ea91e8b155f7b65e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f34b8244a20472ea91e8b155f7b65e5, b03281a52cfa47dd99ccf9db9d55dbcb; no voters:
I20260812 19:08:23.229506 21610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f34b8244a20472ea91e8b155f7b65e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f34b8244a20472ea91e8b155f7b65e5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42001 } } peers { permanent_uuid: "b03281a52cfa47dd99ccf9db9d55dbcb" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 43423 } } peers { permanent_uuid: "f2f6c9b95e7a4ce998922cf4190f2b09" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 34691 } } }
I20260812 19:08:23.229660 21610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f34b8244a20472ea91e8b155f7b65e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:08:23.229755 21610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f34b8244a20472ea91e8b155f7b65e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:08:23.229831 21610 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f34b8244a20472ea91e8b155f7b65e5 [term 1 LEADER]: Becoming Leader. State: Replica: 6f34b8244a20472ea91e8b155f7b65e5, State: Running, Role: LEADER
I20260812 19:08:23.230163 21610 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f34b8244a20472ea91e8b155f7b65e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f34b8244a20472ea91e8b155f7b65e5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42001 } } peers { permanent_uuid: "b03281a52cfa47dd99ccf9db9d55dbcb" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 43423 } } peers { permanent_uuid: "f2f6c9b95e7a4ce998922cf4190f2b09" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 34691 } }
I20260812 19:08:23.232177 21610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f34b8244a20472ea91e8b155f7b65e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f34b8244a20472ea91e8b155f7b65e5. Latest consensus state: current_term: 1 leader_uuid: "6f34b8244a20472ea91e8b155f7b65e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f34b8244a20472ea91e8b155f7b65e5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42001 } } peers { permanent_uuid: "b03281a52cfa47dd99ccf9db9d55dbcb" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 43423 } } peers { permanent_uuid: "f2f6c9b95e7a4ce998922cf4190f2b09" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 34691 } } }
I20260812 19:08:23.232429 21610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f34b8244a20472ea91e8b155f7b65e5 [sys.catalog]: This master's current role is: LEADER
W20260812 19:08:23.232699 21593 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6f34b8244a20472ea91e8b155f7b65e5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:08:23.232779 21593 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:08:23.233807 21620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:08:23.234787 21620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:08:23.235932 21567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2f6c9b95e7a4ce998922cf4190f2b09: No bootstrap required, opened a new log
I20260812 19:08:23.237696 21581 sys_catalog.cc:455] T 00000000000000000000000000000000 P b03281a52cfa47dd99ccf9db9d55dbcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f34b8244a20472ea91e8b155f7b65e5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42001 } } peers { permanent_uuid: "b03281a52cfa47dd99ccf9db9d55dbcb" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 43423 } } peers { permanent_uuid: "f2f6c9b95e7a4ce998922cf4190f2b09" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 34691 } } }
I20260812 19:08:23.237840 21581 sys_catalog.cc:458] T 00000000000000000000000000000000 P b03281a52cfa47dd99ccf9db9d55dbcb [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:08:23.240059 21567 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2f6c9b95e7a4ce998922cf4190f2b09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f34b8244a20472ea91e8b155f7b65e5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42001 } } peers { permanent_uuid: "b03281a52cfa47dd99ccf9db9d55dbcb" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 43423 } } peers { permanent_uuid: "f2f6c9b95e7a4ce998922cf4190f2b09" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 34691 } }
I20260812 19:08:23.240329 21567 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2f6c9b95e7a4ce998922cf4190f2b09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2f6c9b95e7a4ce998922cf4190f2b09, State: Initialized, Role: FOLLOWER
I20260812 19:08:23.240674 21567 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2f6c9b95e7a4ce998922cf4190f2b09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f34b8244a20472ea91e8b155f7b65e5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42001 } } peers { permanent_uuid: "b03281a52cfa47dd99ccf9db9d55dbcb" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 43423 } } peers { permanent_uuid: "f2f6c9b95e7a4ce998922cf4190f2b09" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 34691 } }
I20260812 19:08:23.242552 21534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f34b8244a20472ea91e8b155f7b65e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f2f6c9b95e7a4ce998922cf4190f2b09"
I20260812 19:08:23.242811 21534 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2f6c9b95e7a4ce998922cf4190f2b09 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:08:23.244632 21534 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f2f6c9b95e7a4ce998922cf4190f2b09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f34b8244a20472ea91e8b155f7b65e5 in term 1.
I20260812 19:08:23.242483 21567 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2f6c9b95e7a4ce998922cf4190f2b09 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:08:23.254074 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:23.232481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35750 (local address 127.25.130.62:42001)
0812 19:08:23.232589 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:23.232593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:23.238233 (+ 5640us) server_negotiation.cc:407] Connection header received
0812 19:08:23.238753 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:23.238757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:23.238804 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:23.238850 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:23.240613 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.241335 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.244336 (+ 3001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.244663 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.244748 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:23.247742 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:23.247746 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:23.247748 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:23.247778 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:23.249432 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:23.249939 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:23.249943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:23.249944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:23.250037 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:23.250336 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:23.250339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:23.250341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:23.250566 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:23.250678 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:23.253776 (+ 3098us) server_negotiation.cc:299] Negotiation successful
0812 19:08:23.253911 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 19:08:23.254344 21534 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f2f6c9b95e7a4ce998922cf4190f2b09 [term 1 FOLLOWER]: Refusing update from remote peer 6f34b8244a20472ea91e8b155f7b65e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:08:23.254700 21625 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2f6c9b95e7a4ce998922cf4190f2b09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f34b8244a20472ea91e8b155f7b65e5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42001 } } peers { permanent_uuid: "b03281a52cfa47dd99ccf9db9d55dbcb" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 43423 } } peers { permanent_uuid: "f2f6c9b95e7a4ce998922cf4190f2b09" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 34691 } } }
I20260812 19:08:23.254845 21625 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2f6c9b95e7a4ce998922cf4190f2b09 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:08:23.255635 21610 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6f34b8244a20472ea91e8b155f7b65e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2f6c9b95e7a4ce998922cf4190f2b09" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 34691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 19:08:23.258637 21639 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f2f6c9b95e7a4ce998922cf4190f2b09: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:08:23.258728 21639 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:08:23.259701 21625 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2f6c9b95e7a4ce998922cf4190f2b09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f34b8244a20472ea91e8b155f7b65e5. Latest consensus state: current_term: 1 leader_uuid: "6f34b8244a20472ea91e8b155f7b65e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f34b8244a20472ea91e8b155f7b65e5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42001 } } peers { permanent_uuid: "b03281a52cfa47dd99ccf9db9d55dbcb" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 43423 } } peers { permanent_uuid: "f2f6c9b95e7a4ce998922cf4190f2b09" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 34691 } } }
I20260812 19:08:23.259824 21625 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2f6c9b95e7a4ce998922cf4190f2b09 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:08:23.260829 21570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:23.232195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53590 (local address 127.25.130.60:34691)
0812 19:08:23.232308 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:23.232312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:23.238130 (+ 5818us) server_negotiation.cc:407] Connection header received
0812 19:08:23.238494 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:23.238497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:23.238559 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:23.238612 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:23.242534 (+ 3922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.243214 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.245502 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.245808 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.245881 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:23.255825 (+ 9944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:23.255830 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:23.255832 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:23.255858 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:23.257405 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:23.258310 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:23.258314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:23.258316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:23.258407 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:23.259175 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:23.259178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:23.259180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:23.259412 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:23.259521 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:23.260559 (+ 1038us) server_negotiation.cc:299] Negotiation successful
0812 19:08:23.260687 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 19:08:23.260910 21443 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b03281a52cfa47dd99ccf9db9d55dbcb [term 1 FOLLOWER]: Refusing update from remote peer 6f34b8244a20472ea91e8b155f7b65e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:08:23.261665 21482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:23.233437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38192 (local address 127.25.130.61:43423)
0812 19:08:23.233583 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:23.233588 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:23.247757 (+ 14169us) server_negotiation.cc:407] Connection header received
0812 19:08:23.247944 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:23.247947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:23.248007 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:23.248075 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:23.248913 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.249706 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.250726 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:23.251023 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:23.251095 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:23.252706 (+ 1611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:08:23.252710 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:08:23.252713 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:08:23.252738 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:08:23.254356 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:23.258728 (+ 4372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:23.258732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:23.258734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:23.258836 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:08:23.259833 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:08:23.259837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:08:23.259839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:08:23.260092 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:08:23.260194 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:23.261363 (+ 1169us) server_negotiation.cc:299] Negotiation successful
0812 19:08:23.261523 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 19:08:23.262281 21579 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6f34b8244a20472ea91e8b155f7b65e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b03281a52cfa47dd99ccf9db9d55dbcb" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 43423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:08:23.266906 21581 sys_catalog.cc:455] T 00000000000000000000000000000000 P b03281a52cfa47dd99ccf9db9d55dbcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f34b8244a20472ea91e8b155f7b65e5. Latest consensus state: current_term: 1 leader_uuid: "6f34b8244a20472ea91e8b155f7b65e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f34b8244a20472ea91e8b155f7b65e5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42001 } } peers { permanent_uuid: "b03281a52cfa47dd99ccf9db9d55dbcb" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 43423 } } peers { permanent_uuid: "f2f6c9b95e7a4ce998922cf4190f2b09" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 34691 } } }
I20260812 19:08:23.267058 21581 sys_catalog.cc:458] T 00000000000000000000000000000000 P b03281a52cfa47dd99ccf9db9d55dbcb [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:08:23.272233 21610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f34b8244a20472ea91e8b155f7b65e5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6f34b8244a20472ea91e8b155f7b65e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f34b8244a20472ea91e8b155f7b65e5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42001 } } peers { permanent_uuid: "b03281a52cfa47dd99ccf9db9d55dbcb" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 43423 } } peers { permanent_uuid: "f2f6c9b95e7a4ce998922cf4190f2b09" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 34691 } } }
I20260812 19:08:23.272380 21610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f34b8244a20472ea91e8b155f7b65e5 [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:23.272722 21581 sys_catalog.cc:455] T 00000000000000000000000000000000 P b03281a52cfa47dd99ccf9db9d55dbcb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6f34b8244a20472ea91e8b155f7b65e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f34b8244a20472ea91e8b155f7b65e5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42001 } } peers { permanent_uuid: "b03281a52cfa47dd99ccf9db9d55dbcb" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 43423 } } peers { permanent_uuid: "f2f6c9b95e7a4ce998922cf4190f2b09" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 34691 } } }
I20260812 19:08:23.272856 21581 sys_catalog.cc:458] T 00000000000000000000000000000000 P b03281a52cfa47dd99ccf9db9d55dbcb [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:08:23.273236 21579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f34b8244a20472ea91e8b155f7b65e5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6f34b8244a20472ea91e8b155f7b65e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f34b8244a20472ea91e8b155f7b65e5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42001 } } peers { permanent_uuid: "b03281a52cfa47dd99ccf9db9d55dbcb" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 43423 } } peers { permanent_uuid: "f2f6c9b95e7a4ce998922cf4190f2b09" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 34691 } } }
I20260812 19:08:23.273361 21579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f34b8244a20472ea91e8b155f7b65e5 [sys.catalog]: This master's current role is: LEADER
I20260812 19:08:23.273546 21643 mvcc.cc:205] Tried to move back new op lower bound from 7317756736522121216 to 7317756736434753536. Current Snapshot: MvccSnapshot[applied={T|T < 7317756736522121216}]
I20260812 19:08:23.274401 21625 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2f6c9b95e7a4ce998922cf4190f2b09 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6f34b8244a20472ea91e8b155f7b65e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f34b8244a20472ea91e8b155f7b65e5" member_type: VOTER last_known_addr { host: "127.25.130.62" port: 42001 } } peers { permanent_uuid: "b03281a52cfa47dd99ccf9db9d55dbcb" member_type: VOTER last_known_addr { host: "127.25.130.61" port: 43423 } } peers { permanent_uuid: "f2f6c9b95e7a4ce998922cf4190f2b09" member_type: VOTER last_known_addr { host: "127.25.130.60" port: 34691 } } }
I20260812 19:08:23.274531 21625 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2f6c9b95e7a4ce998922cf4190f2b09 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:08:23.282578 21620 catalog_manager.cc:1347] Generated new cluster ID: 4b41e0627b2f4bfca648b58d1495c66e
I20260812 19:08:23.282672 21620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:08:23.337749 21620 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:08:23.339166 21620 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:08:23.366904 21620 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f34b8244a20472ea91e8b155f7b65e5: Generated new TSK 0
I20260812 19:08:23.367815 21620 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 19:08:23.406666 21570 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:08:23.402964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53600 (local address 127.25.130.60:34691)
0812 19:08:23.403109 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:23.403115 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:23.405351 (+ 2236us) server_negotiation.cc:407] Connection header received
0812 19:08:23.405636 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:23.405639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:23.405702 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:23.405758 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:23.406507 (+ 749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53600: BlockingRecv error: recv got EOF from 127.0.0.1:53600 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260812 19:08:23.408416 21387 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:08:23.403977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35756 (local address 127.25.130.62:42001)
0812 19:08:23.404106 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:23.404111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:23.407104 (+ 2993us) server_negotiation.cc:407] Connection header received
0812 19:08:23.407525 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:23.407540 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:23.407602 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:23.407657 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:23.408277 (+ 620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35756: BlockingRecv error: recv got EOF from 127.0.0.1:35756 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260812 19:08:23.410087 21482 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:08:23.404728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38194 (local address 127.25.130.61:43423)
0812 19:08:23.404844 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:23.404849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:23.407432 (+ 2583us) server_negotiation.cc:407] Connection header received
0812 19:08:23.409061 (+ 1629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:23.409065 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:23.409125 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:23.409175 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:23.409954 (+ 779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38194: BlockingRecv error: recv got EOF from 127.0.0.1:38194 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260812 19:08:23.415153 21482 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:08:23.413373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38200 (local address 127.25.130.61:43423)
0812 19:08:23.413507 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:23.413512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:23.414054 (+ 542us) server_negotiation.cc:407] Connection header received
0812 19:08:23.414300 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:23.414304 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:23.414390 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:23.414443 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:23.415019 (+ 576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38200: BlockingRecv error: recv got EOF from 127.0.0.1:38200 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260812 19:08:23.420889 21482 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:08:23.419143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38208 (local address 127.25.130.61:43423)
0812 19:08:23.419267 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:23.419272 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:23.419846 (+ 574us) server_negotiation.cc:407] Connection header received
0812 19:08:23.420054 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:23.420058 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:23.420116 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:23.420167 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:23.420759 (+ 592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38208: BlockingRecv error: recv got EOF from 127.0.0.1:38208 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260812 19:08:23.427357 21482 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:08:23.425487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38210 (local address 127.25.130.61:43423)
0812 19:08:23.425622 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:23.425627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:23.426256 (+ 629us) server_negotiation.cc:407] Connection header received
0812 19:08:23.426478 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:23.426482 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:23.426558 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:23.426617 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:23.427224 (+ 607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38210: BlockingRecv error: recv got EOF from 127.0.0.1:38210 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260812 19:08:23.435820 21482 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:08:23.433907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38226 (local address 127.25.130.61:43423)
0812 19:08:23.434068 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:23.434073 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:23.434721 (+ 648us) server_negotiation.cc:407] Connection header received
0812 19:08:23.434949 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:23.434954 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:23.435012 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:23.435068 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:23.435689 (+ 621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38226: BlockingRecv error: recv got EOF from 127.0.0.1:38226 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260812 19:08:23.448383 21482 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:08:23.446521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38228 (local address 127.25.130.61:43423)
0812 19:08:23.446661 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:23.446666 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:23.447357 (+ 691us) server_negotiation.cc:407] Connection header received
0812 19:08:23.447595 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:23.447600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:23.447674 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:23.447785 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:23.448250 (+ 465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38228: BlockingRecv error: recv got EOF from 127.0.0.1:38228 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260812 19:08:23.467409 21482 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:08:23.466360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38242 (local address 127.25.130.61:43423)
0812 19:08:23.466503 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:23.466507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:23.466630 (+ 123us) server_negotiation.cc:407] Connection header received
0812 19:08:23.466764 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:23.466767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:23.466816 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:23.466888 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:23.467281 (+ 393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38242: BlockingRecv error: recv got EOF from 127.0.0.1:38242 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260812 19:08:23.502820 21482 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:08:23.501591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38248 (local address 127.25.130.61:43423)
0812 19:08:23.501781 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:23.501786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:23.501917 (+ 131us) server_negotiation.cc:407] Connection header received
0812 19:08:23.502072 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:23.502075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:23.502125 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:23.502196 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:23.502693 (+ 497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38248: BlockingRecv error: recv got EOF from 127.0.0.1:38248 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260812 19:08:23.570856 21482 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:08:23.569423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38250 (local address 127.25.130.61:43423)
0812 19:08:23.569621 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:23.569626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:23.569787 (+ 161us) server_negotiation.cc:407] Connection header received
0812 19:08:23.569950 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:23.569953 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:23.570022 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:23.570102 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:23.570690 (+ 588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38250: BlockingRecv error: recv got EOF from 127.0.0.1:38250 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260812 19:08:23.702864 21482 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:08:23.701503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38254 (local address 127.25.130.61:43423)
0812 19:08:23.701646 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:23.701650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:23.701730 (+ 80us) server_negotiation.cc:407] Connection header received
0812 19:08:23.701888 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:23.701892 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:23.701951 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:23.702041 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:23.702675 (+ 634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38254: BlockingRecv error: recv got EOF from 127.0.0.1:38254 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260812 19:08:23.962582 21482 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:08:23.961341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38258 (local address 127.25.130.61:43423)
0812 19:08:23.961529 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:23.961533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:23.961552 (+ 19us) server_negotiation.cc:407] Connection header received
0812 19:08:23.961722 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:23.961726 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:23.961804 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:23.961897 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:08:23.962425 (+ 528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38258: BlockingRecv error: recv got EOF from 127.0.0.1:38258 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 19:08:24.225625 21607 catalog_manager.cc:1259] Loaded cluster ID: 4b41e0627b2f4bfca648b58d1495c66e
I20260812 19:08:24.225767 21607 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b03281a52cfa47dd99ccf9db9d55dbcb: loading cluster ID for follower catalog manager: success
I20260812 19:08:24.227824 21607 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b03281a52cfa47dd99ccf9db9d55dbcb: acquiring CA information for follower catalog manager: success
I20260812 19:08:24.229204 21607 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b03281a52cfa47dd99ccf9db9d55dbcb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 19:08:24.260596 21639 catalog_manager.cc:1259] Loaded cluster ID: 4b41e0627b2f4bfca648b58d1495c66e
I20260812 19:08:24.260722 21639 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f2f6c9b95e7a4ce998922cf4190f2b09: loading cluster ID for follower catalog manager: success
I20260812 19:08:24.262739 21639 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f2f6c9b95e7a4ce998922cf4190f2b09: acquiring CA information for follower catalog manager: success
I20260812 19:08:24.264104 21639 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f2f6c9b95e7a4ce998922cf4190f2b09: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 19:08:24.482096 21731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:24.477244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38270 (local address 127.25.130.61:43423)
0812 19:08:24.477575 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:24.477579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:24.477677 (+ 98us) server_negotiation.cc:407] Connection header received
0812 19:08:24.477853 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:24.477856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:24.478034 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:24.478161 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:08:24.478502 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:24.479123 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:24.480139 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:24.480416 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:24.480513 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:24.480941 (+ 428us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:08:24.481606 (+ 665us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:08:24.481724 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:24.481805 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 19:08:24.481875 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":95,"threads_started":1}
I20260812 19:08:24.490209 21738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:24.485713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53606 (local address 127.25.130.60:34691)
0812 19:08:24.486084 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:24.486088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:24.486438 (+ 350us) server_negotiation.cc:407] Connection header received
0812 19:08:24.486580 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:24.486582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:24.486735 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:24.486861 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:08:24.487075 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:24.487665 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:24.488520 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:24.488767 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:24.488854 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:24.489091 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:08:24.489732 (+ 641us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:08:24.489839 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:24.489941 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 19:08:24.490003 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":117,"threads_started":1}
I20260812 19:08:24.498020 21738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:24.493948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53614 (local address 127.25.130.60:34691)
0812 19:08:24.494172 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:24.494178 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:24.494482 (+ 304us) server_negotiation.cc:407] Connection header received
0812 19:08:24.494610 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:24.494613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:24.494678 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:24.494756 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:08:24.495042 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:24.495684 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:24.496771 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:24.497060 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:24.497152 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:24.497441 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:08:24.497574 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:08:24.497642 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:24.497756 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 19:08:24.497830 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260812 19:08:24.498836 21731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:24.494256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38272 (local address 127.25.130.61:43423)
0812 19:08:24.494414 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:24.494419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:24.494868 (+ 449us) server_negotiation.cc:407] Connection header received
0812 19:08:24.495079 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:24.495083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:24.495152 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:24.495206 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:08:24.495599 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:24.496298 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:24.497483 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:24.497835 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:24.497933 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:24.498298 (+ 365us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:08:24.498399 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:08:24.498448 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:24.498599 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 19:08:24.498673 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 19:08:24.499192 21744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:08:24.493880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35764 (local address 127.25.130.62:42001)
0812 19:08:24.494362 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0812 19:08:24.494367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:08:24.494407 (+ 40us) server_negotiation.cc:407] Connection header received
0812 19:08:24.494557 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:08:24.494560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:08:24.494820 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:08:24.494963 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:08:24.495160 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:24.495920 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:24.496935 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:08:24.497311 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:08:24.497419 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:08:24.498034 (+ 615us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:08:24.498542 (+ 508us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:08:24.498632 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 19:08:24.498933 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 19:08:24.499021 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":111,"threads_started":1}
I20260812 19:08:24.502986 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 21300
I20260812 19:08:24.517987 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 21391
I20260812 19:08:24.527709 26120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt2JiBs/build/debug/bin/kudu with pid 21484
2026-08-12T19:08:24Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3170 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6468 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (288533 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST